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Abstract 

Concerning bivariate least squares linear regression, the classical ap- 
proach pursued for functional models in earlier attempts (York, 1966; 
1969) is reviewed using a new formalism in terms of deviation (ma- 
trix) traces which, for homoscedastic data, reduce to usual quantities 
leaving aside an unessential (but dimensional) multiplicative factor. 
Within the framework of classical error models, the dependent vari- 
able relates to the independent variable according to the usual additive 
model. The classes of linear models considered are regression lines in 
the general case of correlated errors in X and in Y for heteroscedastic 
data, and in the opposite limiting situations of (i) uncorrelated errors 
in X and in Y , and (ii) completely correlated errors in X and in Y . 
The special case of (C) generalized orthogonal regression is considered 
in detail together with well known subcases, namely: (Y) errors in X 
negligible (ideally null) with respect to errors in Y] (X) errors in Y 
negligible (ideally null) with respect to errors in X; (O) genuine or- 
thogonal regression; (R) reduced major-axis regression. In the limit of 
homoscedastic data, the results determined for functional models are 
compared with their counterparts related to extreme structural mod- 
els i.e. the instrumental scatter is negligible (ideally null) with respect 
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to the intrinsic scatter (Isobe et al., 1990; Feigelson and Babu, 1992). 
While regression hne slope and intercept estimators for functional and 
structural models necessarily coincide, the contrary holds for related 
variance estimators even if the residuals obey a Gaussian distribution, 
with the exception of Y models. An example of astronomical appli- 
cation is considered, concerning the [0/H]-[Fe/H] empirical relations 
deduced from five samples related to different stars and/or different 
methods of oxygen abundance determination. For selected samples 
and assigned methods, different regression models yield consistent re- 
sults within the errors (=Fc) for both heteroscedastic and homoscedas- 
tic data. Conversely, samples related to different methods produce 
discrepant results, due to the presence of (still undetected) systematic 
errors, which implies no definitive statement can be made at present. 
A comparison is also made between different expressions of regression 
line slope and intercept variance estimators, where fractional discrep- 
ancies are found to be not exceeding a few percent, which grows up to 
about 20% in presence of large dispersion data. An extension of the 
formalism to structural models is left to a forthcoming paper. 
keywords - galaxies: evolution - stars: formation; evolution - methods: 
data analysis - methods: statistical. 

pacs codes: 98.62.-g; 97.10.Cv; 02.50.-r 

1 Introduction 

Linear regression is a fundamental and frequently used statistical tool in 
almost all branches of science, such as astronomy, biology, chemistry, geol- 
ogy, physics, and statistics of course; for a full discussion refer to a classical 
paper (Isobe et al., 1990; hereafter quoted as Ial90). In spite of its apparent 
simplicity, the task of drawing the "best" straight line through data on a 
Cartesian plot is difficult and controversial. The problem is twofold: regres- 
sion line slope and intercept estimators are expressed involving minimizing 
or maximizing some function of the data; on the other hand, regression line 
slope and intercept variance estimators are expressed requiring knowledge of 
the error distributions of the data. 

The complexity mainly arises from the occurrence of intrinsic dispersion, 
which could be related to a non Gaussian distribution, in addition to the 
dispersion related to the measurement processes (hereafter quoted as instru- 
mental dispersion), which necessarily implies a Gaussian distribution. An 
increasing difficulty is encountered in more exotic situations, such as trun- 
cated regression, where a variable is assumed to be truncated below or above 
a threshold, and censored regression, where several data are assumed to be 
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undetected at various sensitivity levels. For further details of astronomical 
interest, refer to a classical paper (Feigelson and Babu, 1992; erratum, 2011; 
hereafter quoted together as FB92) and, in general, to specific texts on the 
subject (e.g., Klein and Moeschberger, 2005). 

In statistics, problems where the true points lie precisely on an expected 
line are called functional regression models, while problems where the true 
points are (intrinsically) scattered about an expected line are called struc- 
tural regression models. Accordingly, functional regression models may be 
conceived as structural regression models where the intrinsic dispersion is 
negligible (ideally null) with respect to the instrumental dispersion. A dis- 
tinction between functional and structural modelling is currently preferred, 
where the former can be affected by intrinsic scatter but with no or only min- 
imal assumptions on related distributions, while the latter implies (usually 
parametric) models are placed on the above mentioned distributions. For 
further details refer to specific textbooks (e.g., Carroll et al., 2006, Chap. 2, 
§2.1). In addition, models where the instrumental dispersion is the same from 
point to point for each variable, are called homoscedastic models, while mod- 
els where the instrumental dispersion is (in general) different from point to 
point, are called hctcrosccdastic models. Similarly, related data are denoted 
as homoscedastic and heteroscedastic, respectively. 

Bivariate least squares linear regression related to heteroscedastic func- 
tional models with uncorrelated and correlated errors, following Gaussian 
distributions, were analysed and formulated in two classical papers (York, 
1966; 1969; hereafter quoted as Y66 and Y69, respectively). Bivariate least 
squares linear regression related to extreme structural models, where the 
instrumental dispersion is negligible (ideally null) with respect to intrinsic 
dispersion, was exhaustively treated in a classical paper (Ial90). 

An extension to homoscedastic functional and structural models was per- 
formed in a subsequent paper (FB92), yielding the same expression of regres- 
sion line slope and intercept estimators, provided the instrumental dispersion 
in the former case coincides with the intrinsic dispersion (assumed to be dom- 
inant) in the latter case, for each variable, and the residuals follow a Gaussian 
distribution. 

Further extension to homoscedastic structural models where instrumental 
and intrinsic dispersion are of the same order, was carried in a later paper 
(Akritas and Bershday, 1996; hereafter quoted as AB96). Heteroscedastic 
structural models with instrumental dispersion negligible with respect to 
intrinsic dispersion in one variable, were also presented (AB96). 

The above mentioned papers provide the simplest description of linear 
regression. More sophisticated attempts imply additional effects such as 
truncated and censored regression (e.g., FB92), analytical methods such as 
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correction of the observed moments of the data (e.g., Fuller, 1987; AB96; 
Freedman et al., 2004), minimization of an effective statistic (e.g., Clutton- 
Brock, 1967; Barker and Diana, 1974; Press et al., 1992; Tremaine et al., 
2002), assuming a probability distribution for the true independent variable 
values (e.g., Schafer, 1987, 2001; Roy and Banerjee, 2006), computational 
methods such as bootstrap and jackknife (e.g., FB92), matrix formalism (e.g., 
Schwarzemberg-Czerny, 1995; Branham, 2001), simultaneous adjustement 
(e.g., Pourbaix, 1998), and Bayesan approach (e.g., Zellner, 1971; Gull, 1989; 
Dellaportas and Stephens, 1995; Carroll et al., 1999; Scheines et al., 1999; 
Kelly, 2007). 

The last investigation is particularly relevant in that it is the first ex- 
ample, in the astronomical literature, where linear regression is considered 
following the modern (since about half a century ago) approach based on 
likelihoods rather than the old (up to about a century ago) least-squares ap- 
proach. More specifically, a hierarchical measurement error model is set up 
therein, the complicated likelihood is written down, and a variety of mini- 
mum least-squares and Bayesan solutions are shown, which can treat func- 
tional, structural, multivariate, truncated and censored mesaurement error 
regression problems. 

Even in dealing with the simplest homoscedastic (or heteroscedastic) func- 
tional and structural models, still no unified analytic formalism has been de- 
veloped (to the knowledge of the author) where (i) structural heteroscedastic 
models with instrumental and intrinsic dispersion of comparable order in 
both variables, are considered; (ii) previous results are recovered in the limit 
of dominant instrumental dispersion; and (iii) previous results are recovered 
in the limit of dominant intrinsic dispersion. A related formulation may be 
useful also for computational methods, in the sense that both the general 
case and limiting situations can be described by a single numerical code. 

The current paper aims at making a first step towards a unified ana- 
lytic formalism of bivariate least squares linear regression involving functional 
models. More specifically, earlier attempts shall be reviewed and reformu- 
lated by definition and use of deviation (matrix) traces, within the framework 
of classical error models where the dependent variable relates to the inde- 
pendent variable according to the usual additive model. 

Homoscedastic and heteroscedastic functional models are presented in 
section 121 basing on two classical papers (Y66; Y69). An example of astro- 
nomical application is outlined in section |3l The discussion is performed in 
section |H Finally, the conclusion is shown in section Some points are 
developed with more detail in the Appendix. An extension of the formalism 
to structural models is left to a forthcoming paper. 
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2 Least-squares fitting of a straight line 



2.1 General considerations 

Attention shall be restricted to the classical problem of least-squares fit- 
ting of a straight line, where both variables are measured with errors and the 
true points lie on the unknown regression line i.e. functional models (e.g., 
Y66; Y69), which can be considered as structural models in the limit of neg- 
ligible (ideally null) intrinsic scatter. In general, the dependent variable, 
relates to the independent variable, x, according to the usual additive model 
(e.g., AB96; Carroll et al., 2006, Chap. 1, §1.2, Chap. 3, §3.2.1; Kelly, 2007; 
Buonaccorsi, 2010, Chap. 4, §4.3): 

ysi = axsi + h + ei ; l<z<n ; (1) 

where Pg^ = {x^i^ysi) are the actual points whose coordinates are affected by 
no instrumental error and is a random variable with null expectation value 
representing the intrinsic scatter in {xsi,ysi) about the regression lin^. 

Due to the occurrence of instrumental errors, the observed points, Pj = 
{Xi,Yi), are evaluated in place of the actual points, Pgj. The coordinates of 
observed and actual points are assumed to be related as: 

= xsi + i^pji ; 1 <i <n ] (2a) 
Y,=ys^ + i^Fy)^ ; l<t<n ; (2b) 

where (^f^^,)*, {^Fy)i, are the instrumental errors on xsi and ysi, respectively, 
assumed to be normally distributed with null expectation values and known 
variances, {(T^^)i = [(c^^)^]^ {cryy)i = [{<7y)i]'^, and covariance,(a^,j/)i = {ay^)i. 
The terms "independent variable" and "dependent variable" are purely con- 
ventional when the model is symmetric in x and in y provided a ^ 0. For 
a vanishing intrinsic scatter, — )■ 0, 1 < i < n, actual points lie on the 
unknown regression line whereas adjusted points, Pj = (xj,?/,), lie on the 
estimated regression line: 

yi = dxi + b; 1 < i < n ; (3) 

where, in general, estimators are denoted by hats. For further details refer 
to earlier attempts (York, 1967; Y69). 

In the case under discussion, the regression estimator minimizes the sum 
(over the n observations) of squared residuals (e.g., Y69), or statistical dis- 
tances of the observed points, Pj = (Xj, Yj), from the estimated line in the un- 
known parameters, a,b,xi, ...,Xn (e.g.. Fuller, 1987, Chap. 1, §1.3.3). Under 

^The Italian convenction shall be adopted here, according to which the slope and the 
intercept of a straight line on the Cartesian plane, are denoted as a, b, respectively. 
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restrictive assumptions, the regression estimator is the functional maximum 
hkehhood estimator (e.g., Carroll et al., 2006, Chap. 3, §3.4.2). 

To the knowledge of the author, only classical error models are consid- 
ered for astronomical applications, and for this reason different error models 
such as Berkson models and mixture error models (e.g., Carroll et al., 2006, 
Chap. 3, Sect. 3.2) shall not be dealt with in the current attempt. From 
this point on, investigation shall be limited to functional models and least- 
squares regression estimators for the following reasons. First, they are im- 
portant models in their own right, furnishing an approximation to real world 
situations. Second, a careful examination of these simple models helps for 
understanding the theoretical underpinnings of methods for other models of 
greater complexity such as hierarchical models (e.g., Kelly, 2007). 



2.2 Functional models 

With regard to functional models, bivariate least squares linear regression 
were analysed in two classical papers (Y66; Y69). The same line of thought 
shall be followed here and the sole changes shall be concerned with the for- 
malism, as clearly indicated. The general case shall first be presented, while 
special cases shall be deduced later as limiting situations. 

In the light of the model outlined in subsection [2TT] in absence of intrinsic 
scatter, Eqs. ([2]) and ([3]), the actual points, Pgj = {xsi,ysi) coincide with the 
true points, P* = {x*,y*), whose coordinates lie on the unknown regression 
line: 

y* = ax* + b ; l<i<n ; (4) 

while the observed points, = {Xi,Yi), are scattered with respect to the 
regression line. 

The coordinates, {xi,yi), may be conceived as the adjusted values of re- 
lated observations, (Xj,Fj), on the calculated regression hne (Y66; Y69), 
Eq. ([2]) and, in addition, as estimators of the coordinates, {x*,y*), on the 
true regression line determined in absence of mesaurement errors, Eq. (jl]). 

The line of adjustment, PjPj (e.g., Y69), may be conceived as an estimator 
of the statistical distance, PjP* (e.g.. Fuller, 1987, Chap. 1, §1.3.3), where 
Pj(xj, yi) is the adjusted point on the estimated regression line and P*(x*, y*) 
is the true point on the true regression line. 

The squared weighted residuals are defined as (Y69): 

^5 ^2 _ - ^i)^ + Wy^0^i - Vif - '^n^w^^Wy^Xi - Xi){Yi - yi) 

[Ki) — 2 ) l^^J 

1 — r- 

^"""•^^^ ■ |r,|<l ; l<t<n ; (5b) 
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where , Wy- , are the weights of the various measurements (or observations) 
and Tj the correlation coefficients. An equivalent formulation in matrix for- 
malism can be found in specific textbooks, where weighted true residuals are 
conceived as "statistical distances" from data points to related points on the 
regression line [e.g.. Fuller, 1987, Chap. 1, §1.3.3, Eq. (1.3.16)]. 
In the limit of uncorrelated errors, Eq. ^ reduces to (Y66): 

{Rif = w^^Xi - Xif + Wy^Yi - Vif ; (6a) 
= ; l<i<n ] (6b) 

where the covariances are necessarily null, {(Txy)i = 0, 1 < i < ra. 

In the limit of perfectly correlated errors, r, = sgn(rj), 1 < i < n, it can 
be seen that the following relation holds: 



V / \ 1/2 



sgn(ri) ; (7) 



where sgn is the sign functioiJl. Accordingly, Eq. fl5a|) reduces to: 

WxiiXi -XiY + Wo,i{Xi -Xi)"^ sgn2(ri) 



{R,y = hm 

ri->-sgn(ri} 



w^X^i-Xi) lim 



1 -r2 

2rjSgn(ri)w^^(Xj - Xj^ 
1 - r2 

2 2 - iTi sgn(ri) 



>sgn(r,) 1 - r1 sgn2(rj) 
2 

= WxX^i-x,f lim — — ; 

ri^sgn(ri) 1 + r j sgn(ri) 

which, owing to Eq. ([7]), takes the form: 

{R,f = w.,XX^-Xif = WyXy^-y^^^ ; (8a) 
Ti = sgn(rj) ; 1 < i < n ; (8b) 

where the covariances are necessarily equal to the (positive or negative) 
square root of the variance product, {(Jxy)i = [{o'xx)iicryy)iY^'^ , 1 < i < n. 

Turning back to the general case, let the squared residual matrix be de- 
fined as: 

= 1 1 RiRj II; l<i<n ; I < J < n ; (9) 



^The sign function is defined as sgn(x) — |a;|/x, x ^ 0; sgn(a;) = 0, a; = 0. 
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which is a square matrix of order, n. Let the related trace: 

n 

Tn = mRi)" ; (10) 



i=l 



be defined as the squared residual trace. The regression estimator is that 
minimizing the squared residual trace (Y66; Y69): 



T^{xi,X2, x„, yi, y2, ...,yn) = Y^i^i 



E 



w^^X^i - ^if + 'Wy^iYi - yif - 2ri^w^^WyXXi - Xi){Yi - yi) 



with the constraint expressed by Eq. ([3]) where the coefficients, a, 6, are still 
to be determined, and for this reason are denoted as a, 6, respectively. If 
the values, (xi, X2, a;„, j/i, y2, yn, a, b), relate to a constrained extremum 
point, then the following relations must necessarily hold (Y66; Y69): 



i=l 



w^^Xj - Xi)Sxi + Wy^Yj - yi)5yi 
ri^/w^lE;;i[{Y^ - yi)5xi + {Xi - Xi)Syi 



6yi — a6xi — Xi6a — 6b = 



l-r2 

1 < i < n 



; (12) 
(13) 



where Eq. ( I13p is also valid after inserting on both sides a multiplier (to be 
specified later), Xi, 1 < i < n. The sum of the ensuing relations yields: 



YXi6yi-aYXi6xi-YXiXi6a-YXi6b = ; 

1=1 i=l 1=1 i=l 

and the sum of the left-hand sides of Eqs. ffT^ and ffn|) produces: 

'w^^^Xj - Xj) - rj^w^^Wy^Yj - yi) 



(14) 



E 

i=l I 



+ E 

1=1 



1 -r2 

WyXYi - yi) - ri^w^^WyXXi - Xi) 
1^71 



aXi 
+ X 



Sxi — ^ XiSb 

i=l 
n 

Syi -^XiXiSa = ; (15) 



i=l 



which implies each coefficient is null, as: 



- Xi) - ' Y "'/' (y. - y^) - a\ = ; 



1 — r, 

w. 



l-r4 



T^(r. - y.) - '-^p^iX, - X.) + A. = 
1- rf 1 - rf 



(16a) 
(16b) 



EA« = o ; (17) 

i=l 
n 

J2\iXi = ; (18) 



for 1 < i < n. 

The combination of Eqs. fll6ap and (116bp yields: 



Xi- Xi= (a - ^) — ; (19a) 



Qi = ; l<i<n ; (20) 

V w^^i 

and the combination of Eqs. ([3]) and (|T9i) produces: 

Ai = Wi{aX, + b-Yi) ; 1 < i < n ; (21) 

finally, the substitution of Eq. f l2T]) into f|T71) and f|T8l) yields, in the latter 
case after some algebra: 

n 

J2W,iaX, + b-Y^ = ; (23) 

i=l 

n n Ti/2 

J2 W^XiiaX, + b-Yi}-aJ2^{aXi + b- r,)^ 

i=l 1=1 "^^i 

+ E— 7f(«^^ + ^-^^)' = o ; (24) 

i=l 

where the regression line slope and intercept estimators, d and b, are found 
solving the system of Eqs. (123!) and (^^. 
In terms of the weighted means: 

n 

Z = i4 ; Z = X,Y ; (25a) 

i=l 

n 

Y,Wi{Zi-Z) = ; Z = X,Y ; (25b) 

i=l 



the intercept is expressed by casting Eq. fl2^ under the equivalent form: 

b = Y-aX ; (26) 

the point, P = {X,Y), clearly lies on the regression line, and can be con- 
sidered as the "barycentre" of the data, Pj = 1 < i < n (Y66; 
Y69). 

Using Eq. fl26|) . the following relation holds: 

(27) 



aXi + b-Yi = a{Xi -X)-{Yi-Y) ; l<i<n 



in terms of the deviations from the weighted mean, (Xj — X) and (Kj — Y). 
Using Eqs. ([23]), ([26]) and ([27]), together with the identities, Zi = {Zi-Z) + Z, 
l<i<n, Z = X,Y, the following relations are found after some algebra: 

n n n 

WiX,{aXi + h-Yi)=aY^W,{X,-Xf-Y^ W,{X, - X)(F, - r);(28) 



i=l 



Y: V,{aX, + b - Y,f = a^Y. ^^(^^ " + E ^^(^^ - 



i=l 



i=l 



i=l 



2aYV^{X^~X){Y:,-Y) 



(29) 



i=l 



Y HaX^ + b - Yi)^ = a^J2 U^{X^ -Xy + Y U^{Y^ " Yf 



i=l 



i=l 



2aJ2U^iX,~X){Y,-Y) 



i=l 

(1 + a2(]2 _ 2ar,fi,)2 



1 < i < n ; 
1 < i < n 



Wx^Qi (1 + a2^]2 _ 2arA)2 ' 
and the substitution of Eqs. ([28]), ([29]), ([30]), into ^ yields (Y69): 



(30) 
(31) 
(32) 



Yv.ix,-xf 



i=l 



2 J2 V^iX, - X)(Y, -Y) + Y U^{X, - Xf 



i=l 



—a 



J2 W^{X, -Xf-Y ViiY, -Yf-2Y Ur{X, - X){J, - Y) 



+ 



.i=\ 
n 



i=l 



Y W,{X, - X){Y, - F) - E U^{Y^ - Y) 



i=l 
2 



; 



(33) 



where the resulting terms have been ordered in decreasing powers of the slope, 
a, and the coefficients, Wi, Vi, Ui, 1 < i < n, depend in turn on the slope, as 
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shown by Eqs. fl2^ . ([SI]), ([32]), respectively. Then Eq. (15^ is a pseudo cubic 
equation which can be iteratively solved with the desired degree of precision, 
provided Wi, Vi, Ui, 1 < i < n, are weakly dependent on a. 

With the aim of getting a more compact formalism, let the weighted 
deviation matrices be defined as: 

l<i<n; l<j<n; (34) 
which are square matrices of order, n. Let the related traces: 

n 

Qpg = J2Q^iX^-Xyi'^^-^y ! Q = W,V,U,P ; (35) 

i=l 

be defined as the weighted deviation traces. Pure and mixed traces occur 
for p = and/or q = 0; p > and q > 0; respectively. The special cases, 
(P) q) = (2, 0), (0, 2); (1, 1); relate to expressions used in earlier attempts for 
both weighted (Y66; Y69) and unweighted (Ial90; FB92) residuals. The 
special case, {p,q) = (0,0), yields the product, nQ. With this notation, 
Eq. reads: 

V2oa^ - i2Vn + U2o)a^ - (W^o - V02 - 2Un)a + (Wn - f/02) = ; (36) 
and the solutions can be determined along the following steps. 

(1) Estimate the slope, a^^\ of the regression line, and calculate related 

deviation traces, Q^^, appearing in Eq. ( 15B]) . 

(2) Solve the corresponding cubic equation, and select the solution of inter- 

est, a^'^\ 

(3) Check the inequality, |a*-*Ya*^*~^'' — 1| < v, where i = 2 and v is the 

desired degree of precision. 

(4) If the above inequality is not satisfied, return to (1) using a*^^^ instead of 

a^^\ or in general a^^~^^^ instead of a^^\ 

(5) If the above inequality is satisfied for i = n, then d = a^"^^ is the regression 

line slope estimator. As pointed out in the parent paper (Y66) and 
confirmed by the author, in all cases considered three real solutions are 
found, where the one of interest coincides with the third appearing in 
standard formulae. For further details refer to Appendix lAl 



M; 



Q,QjiX,-XyiY,-YY 
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The pseudo cubic, Eq. fl5^ . may be reduced algebrically to a pseudo quadratic 
equation (Y69) which, in terms of deviation traces, expressed by Eq. fl5^ via 
Eqs. (13T]) and (l32l) . may be cast under the form: 



(^11 - f/2o)a' + (i^2o - Vo2)a - (Pn - f/02) 







1 < z < 72 



(37) 
(38) 



where no explanation and no quotation are provided in the parent paper 
(Y69) on how Eq. (137|) can be deduced from Eq. (l36ll . Surely, the solution of 
interest must necessarily be chosen among the three of the pseudo cubic or 
the two of the pseudo quadratic. 

At this stage, the regression line slope and intercept estimators, a, and 
6, can be determined via Eqs. fl36|l or fl371) and fl26|l . respectively. An exact 
expression of the regression line slope and intercept variance estimators may 
be calculated using the method of partial differentiation, along the following 
steps (Y69). 

(1) Cast the pseudo quadratic equation, Eq. fl371) . into the implicit form: 

0(Xi,ri,a) = A(Xi,y;,a)(a)2 + P(Xi,ri,a)a + C(Xi,ri,a) = ;(39a) 



^(X„F„a) = \/n-f/2o 
P(X„F„a) = P2o-K,2 
C{Xi^ Yi, a) = U02 — Pn 

where Zi stands for Zi, Z2, Zn, and Z = X,Y. 



(39b) 
(39c) 
(39d) 



(2) Write the quadratic error propagation formula related to weighted and 
correlated measurements: 



E 

i=l 



+ 



1 



+ 



2rj d(j) d(f) 



w 



Xi 



dViJ Wy^ ' y/w^^Wy^ dXi dYi 



.(40) 



(3) Calculate the explicit expression of the partial derivatives, d(f)/ da, dip/dXi, 
d(p/dYi, 1 < i < n, using Eqs. (E2]), ([35]), (|38]), and 



(4) Calculate the regression line slope variance estimator, (aa)^, using Eq. ({40 
and multiplying by the squared residual trace, using Eq. (|TT]) . and di- 
viding by (n — 2), as: 



era 



1^T~ 
n-2 ^ 



(41) 
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(5) Cast the regression line intercept estimator, expressed by Eq. ( EBjl . into 

the imphcit form: 

b = ^{X„ Y,) = Y- d{X„ Yi)X ■ (42) 
where Zi stands for Zi, Z2, Zn, and Z = X,Y. 

(6) Write the quadratic error propagation formula related to weighted and 

correlated measurements: 



E 



dtp \ 1 ( dil)\'^ 1 2rj di/j difj 



dXi) w^, \dYi) Wy^ ^/WxiWy^ dXi dYi 



■ (43) 



(7) Calculate the explicit expression of the partial derivatives, dip/dXi, 
dip/dYi, I < i < n, using Eqs. and the theorem on the 

derivative of a function of a function: 

d(i) del) da , , 

Z = X,Y ■ l<i<n . (44) 



dZi da dZi 

(8) Calculate the regression line intercept variance estimator, (ag)^, using 
Eq. fH^ and multiplying by the squared residual trace, using Eq. flTT]) . 
and dividing by (n — 2), as: 

- . (45) 

The calculation of the partial derivatives, d(l)/dd, d(j)/dXi, d(j)/dYi, and 
dtp/dXi, dtp/dYi, 1 < i < n, is performed in Appendix [Bl 

Reasonable approximate values of the regression line slope and intercept 
variance estimators, are expressed as (Y66; Y69)|§: 

i^^? = ^- ; (46) 

'Y.w^{x,-xf 

i=l 



E^*(^* 

1 

n 



hf = (^^f'^n ; (47) 



i=l 



^ The numerator of the fraction in Eq. (1471) has been omitted and put equal to unity 
in the parent paper (Y69) due to a printing error, as it can be argued by considering the 
physical dimensions on both sides, or by comparison with its counterpart in an earlier 
attempt (Y66). 
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which, using Eqs. fl2^ . (^^, and determining the exphcit expression of the 
squared residual trace, may be cast under the equivalent form: 



ar + 



W, 



02 



n 



2a- 



20 



W- 

w, 



11 



20. 



(48) 
(49) 



for a formal demonstration of Eq. (148 p refer to Appendix O 

Relevant and useful special cases shall be discussed in the following sub- 
sections. 



2.3 Uncorrelated errors in X and in Y 

In the limit of uncorrelated errors, {<Jxy)i = 0, = 0, 1 < i < n, Eqs. 
dSH), 032]), and dM} reduce to: 

Ui = ; l<i<n ; (52) 

accordingly, the pseudo cubic, Eq. (jM]), reduces to: 

V2oa^ -2Viia^ -{W2o-Vo2)a + Wn = ; (54) 
and the pseudo quadratic, Eq. (I57|) . reduces to: 

V^iia^ + (P20 - V'o2)a - Pn = ; (55) 



where Upg = via Eqs. ([25]) and 

Following the same procedure outlined in the general case, the regression 
line slope and intercept estimators, a and b, and the regression line slope and 
intercept variance estimators, ((3"a)^ and (o"^)^, can be determined. For further 
details refer to the parent papers (Y66; Y69). A pictorial illustration of the 
method may be found in an additional paper (York, 1967). For a different 
but equivalent approach refer to an independent investigation (Mclntyre et 
al., 1966). An earlier method implying approximate expressions is outlined 
in a pioneering attempt (Deming, 1943). 
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2.4 Completely correlated errors in X and in Y 

In the limit of completely correlated errors, {(Jxy)i = [{(^xx)i{(^yy)i\^^'^ 
(where the positive and the negative root relate to correlation and anticor- 
relation, respectively), = sgn(rj), 1 < i < n, Eqs. fllQap and (]19b|) reduce 
to: 

Xi-Xi = [aVti - sgn(ri)]^^ ; (56a) 
Yi-yi = sgn(ri)[afii - sgn(ri)]— ; (56b) 



Wy. 



and the combination of Eqs. fl56ap and fl56bp yields: 

Yi - yi sgn(ri) 



(57) 



which is equivalent to Eq. ([7]) via Eq. fl20|) . Accordingly, Eq. f|TT]) reduces to: 

n n 

TJi = Y.^^AX^-x,f = J2wyXY,-y,f ; (58) 

1=1 i=l 

and the repetition of the procedure used in the general case yields again 
Eqs. ([2I])-([21, (126])-([33D, dSSD-dM]), where = sgn(ri), I < i < n, when 
necessary. In particular, Eqs. (12^ . (ETl) . and (1551) reduce to: 

sgn(rj) , , 

[afii - sgn(ri)]4 

w Q'^ 
[afii - sgn(ri)]4 

while the pseudo cubic and the pseudo quadratic maintain the formal ex- 
pression of the general case, Eqs. and (jSTj), respectively. For a pictorial 
illustration of the method refer to the parent paper (Y69). 



2.5 Errors in X negligible with respect to errors in Y 

In the limit of errors in X negligible with respect to errors in Y, a'^{axx)i ^ 
{cryy)i, a{axy)i < {<yyy)u I < i < n. Ideally, {axx)i 0, {(Txy)i 0, 
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1 < i < n, which imphes rj — )■ 0, w^.. — )■ +00, i7j — )■ 0, 1 < z < Accord- 
ingly, the errors in X and in Y are uncorrelated. 

In the hmit, w^^ — )■ +00, I < i < n, Eqs. (I50|) - (l53|) . with due account 
taken of Eq. (EO]), reduce to (Y66): 

W^ = Wy^ ; l<i<n ; (63) 

Vi = ] l<i<n ] (64) 

Ui = ; l<i<n ; (65) 

Pi = Wy^; 1 < i <n ; (66) 

accordingly, the pseudo cubic, Eq. (Ell), the pseudo quadratic, Eq. flS^ . 
by use of Eq. ( 135|) reduce to: 

{wy)2oa - {wy)u = ; (67) 

and the regression line slope estimator reads: 

av = ; (68) 



finally, the substitution of Eq. fl68|) into fl26|) yields the regression line inter- 
cept estimator, as: 

= f - ayX ; (69) 

where the index, Y, stands for WLS(Y|X) i.e. weighted least square regres- 
sion, or in particular OLS(Y|X) i.e. ordinary least square regression, of the 
dependent variable, Y, against the independent variable, X (Ial90). 

With regard to the regression line slope and intercept variance estimators, 
(a^^)^ and {ct'^Y, with no account taken of the scatter of the data points, 
Pj = (Xj, Yi), about the regression line, in the case under discussion Eqs. ( HUj) 
and (113]) reduce to: 



cr. 



day J ' ^ \dYiJ wy^ 



E — ; (70) 



[a 



■I ^2 



and the substitution of Eqs. (l208bD . (I209D . and (l2T0bD into (|70l) and (|7T|), 
respectively, yields: 

('^iJ' = 7^ ; (72) 



^ + (Xf 
(^Joo 



(^;j'(^^) ; (73) 
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where Eqs. fl2^ and (1551) have also been used. 

The squared residual trace, expressed by Eq. 
consideration via Eqs. ( 135|) and (163|) reduces to: 

(aY)^(lSy)20 + {ufy)o2 - 2dY{Wy)u 



R 



in the case under 



(74) 



and concerning the regression line slope and intercept variance estimators, 
(o"aY)^ and ((Tj,^)^, with due account taken of the scatter of the data points, 
Pj = (Xj, Yi), about the regression line, in the case under discussion Eqs. ( HT]1 
and (|15]), by use of ([63]), ([72]), ([73]) and ([711) reduce to: 



n-2 



iaYY + 



[a. 



aY ) 



[Wy)Q2 [Wy)ll 
[Wy)2Q 



(75) 
(76) 



where Eqs. fl^ and fl55]) have also been used. It can be seen that, in the 
case under consideration, the exact expression of the regression line slope 
and intercept estimators, Eqs. (1751) and (176]) . coincide with the approximate 
expression in the general case, Eqs. (l48l) and ([19]), respectively. 

The expression of the regression line slope and intercept estimators and re- 
lated variance estimators, Eqs. ([SH]), (1^ . (|751) . ([7^ . coincide with their coun- 
terparts determined for WLS(Y|X) models in a recent attempt [Lavagnini and 
Magno, 2007, Eqs. (17)-(21) therein]. 

The substitution of Eq. ([68]) into ([75]) yields: 

{(Ta 



'ay 



{Wy)02{Wy)20 - [{Wy)u] 



(77) 

(78) 



where D^^ is the determinant of a (weighted) deviation trace matrix, defined 
as a weighted deviation determinant. 



Under the restriction of homoscedastic data, = Wx, Wy^ 



Wy, 1 < 



i < n, which implies Qi = Q, Q 
Eqs. ([25]) and ([35]) reduce to: 



W, V, U, P, via Eqs. ([22]), (|3T]), ([32]), ([MD, 



z = z 

n 



z) 



z - 

-- 



(79a) 
(79b) 



i=l 



Qpq QSp 



ipq — ^^pq 1 
n 



Q = W,V,U,P ; 



1=1 
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where Spg are the (unweighted) pure {p = and/or q = 0) and mixed {p > 
and q > 0) deviation traces, and 5*00 = n. 

In the special case under discussion, Eqs. (168|) . (l69i) . (175|) . fl76l) . (1771) and 
reduce to: 



ay 



S20 

by = Y — CLyX ; 
\2 



1 



cr, 



77, 



CT; 



n2 , 'S'02 5*11 

(ay) +^ 2aY — 

'-'20 '-'20 

-^20 + (Xf] = (a, 



n 



n - 2 (5n)2 



-Ds — 5'o25'2o — (5' 
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(82) 

(83) 
(84) 

(85) 

(86) 
(87) 



where is the determinant of the (unweighted) deviation trace matrix, 
defined as the (unweighted) deviation determinant. 

The expression of the regression line slope variance estimator, Eq. (184|) . 
coincides with earlier results known in literature in the special case of normal 
residuals [e.g., FB92, Eq. (4) therein, in the limit = {o'yy).^^^/ {a xx)7.z7. 
+00, where zzz = ins, int, denote instrumental and intrinsic scatter, respec- 
tively]. For a formal demonstration, see Appendix|Dl Then the expressions of 
the regression line slope and intercept variance estimators, {cFclyY and (cr^^)^, 
reported above, hold provided the residuals obey a Gaussian distribution, as 
expected for functional models (Y66; Y69). 

The expression of the regression line slope and intercept estimators and 
related variance estimators, Eqs. flS^ . f l5^ . f lM|) . fl5^ . coincide with their 
counterparts determined for OLS(Y|X) models in a recent attempt [Lavagnini 
and Magno, 2007, Eqs. (3)- (7) therein]. 



2.6 Errors in Y negligible with respect to errors in X 

In the limit of errors in Y negligible with respect to errors in X, {o'yy)i ^ 
a'^{<yxx)i, {<^xy)i < a{axx)i, I < i < n. Ideally, {ayy)i 0, {(Jxy)i 0, 
1 < i < n, which implies — )■ 0, Wy^ — )■ +00, fli — )■ +00, 1 < i < n. 
Accordingly, the errors in X and in Y are uncorrelated. 

The model under discussion can be related to, but not confused with, the 
inverse regression, which has a large associate literature (e.g.. Miller, 1966; 
Garden et al., 1980; Osborne, 1991; Brown, 1993; Lavagnini and Magno, 
2007). More specifically, the inverse regression consists in the obtainement 
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of a variable, x, from an instrumental response, with the confidence inter- 
val for the true value of x (e.g., Brownlee, 1960; Lavagnini and Magno, 2007). 
A statistical calibration problem is a kind of inverse prediction, a problem 
of retrospection, and some authors call it inverse regression rather than cal- 
ibration: it is probably best explained by considering a typical univariate 
calibration problem (Osborne, 1991). 

In the limit, Wy^ +oo, I < i < n, Eqs. with due account 

taken of Eq. (EO]), reduce to (Y66): 

Wi = a-^w,^ ; l<t<n ; (88) 

Vi = a-'^w^. ; 1 < z < n ; (89) 

Ui = ; l<i<n ; (90) 

p. = ; l<i<n ; (91) 

accordingly, the pseudo cubic, Eq. and the pseudo quadratic, Eq. (13^ . 
by use of Eq. f l35|l reduce to: 

{wx)iia - {wx)o2 = ; (92) 

and the regression line slope estimator reads: 

ax = ; 93 

finally, the substitution of Eq. (jHS]) into fl2B]) yields the regression line inter- 
cept estimator, as: 

bx = Y-dxX ; (94) 

where the index, X, stands for WLS(X|Y) i.e. weighted least square regres- 
sion, or in particular OLS(X|Y) i.e. ordinary least square regression, of the 
independent variable, X, against the dependent variable, Y (Ial90). 

With regard to the regression line slope and intercept variance estimators, 
(o"^^)^ and (o"^ )^, with no account taken of the scatter of the data points, 
Pj = (Xj, Yi), about the regression hne, in the case under discussion Eqs. f HU]) 
and (jinj reduce to: 



(95) 
(96) 
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and the substitution of Eqs. dnZiD, fl^ . and (EHiD into and fl^ . 
respectively, yields: 



[a 



(Wa;)02 



where Eqs. and (1551) have also been used. 

The squared residual trace, expressed by Eq. 
consideration via Eqs. fl35|) and fl88|) reduces to: 



(97) 

; (98) 
in the case under 



(wx)2o + (ax) ^(wx)o2 - 2(ax) ^(wx)ii 



(99) 



and concerning the regression line slope and variance estimators, ((3"ax)^ and 
{cTf^^Y, with due account taken of the scatter of the data points. Pi = (X,, Yi), 
about the regression line, in the case under discussion Eqs. fHT]) and fH5l) . by 
use of dHSD, dnZD. (inSD and reduce to: 



[ax 



n 



[ax 



,2l^2:)20 



(ax)2 



lW^xj02 
('W^a;)o2 , \2/'^n2 



2ax 



>x) 



11 



lW^j02 



{axYixy 



(100) 
(101) 



where it can be seen that, in the case under consideration, the exact ex- 
pressions of the regression line slope and intercept estimator, Eqs. fllOOp and 
(llOip . respectively, do not coincide with the approximate expression in the 
general case, Eqs. (HSj) and ( H^ . respectively. 
The substitution of Eq. ([93]) into ffTOOD yields: 



[a, 



^ {ax? 

{Wx)02{ufx)20 - [(^i^)ll]^ 



(102) 

(103) 



w 



X, ^ ^ i ^ n, which 



where Dy^^ is a weighted deviation determinant. 

In the special case of homoscedastic data, Wx 
implies Qi = Q, Q = W, V, f/, P, via Eqs. (El]), (EI]), ([32]), ([Ml), and Eqs. ((79]) 
and dHQ]) hold. Accordingly, Eqs. ([93]), (jM]), (fTnOjl . flTOTjl . and fiT02|) reduce 
to: 



ax 



02 



(104) 



11 



20 



bx = Y- axX 



'ax. 



n - 2 



'-'02 '-'02 



[cr. 



a. 



ax. 



[ax) 



-'So2 + (ax)'(X)^ 
n 



n - 2 (5n)2 



(105) 
(106) 

(107) 

(108) 



where Ds is the deviation determinant, Eq. 

The expression of the regression hne slope variance estimator, Eq. (11061) . 
is different from its counterpart calculated in an earlier attempt (FB92), due 
to the lack of an additional term which is negligible for Ds/iSn)"^ ^ 1 and/or 
n ^ 1. For a formal demonstration, see Appendix [Dl Then the expressions of 
the regression line slope and intercept variance estimators, ((3"ax)^ and (o"?,^)^, 
reported above, hold provided the residuals obey a Gaussian distribution, as 
expected for functional models (Y66; Y69), with the caveat due to the above 
mentioned discrepancy. 



2.7 Generalized orthogonal regression 

In the limit of constant y to x variance ratios and constant correlation 
coefficients, the following relations hold: 

M^ _^2. ^-2 ^ ^2 . 1^ = r,c = TC ' l<i<n ; (109) 



-'Vi 

where the weights are assumed to be inversely proportional to related vari- 
ances, oc ll{azz)i, z = x,y, as usually done (e.g., FB92). It is worth 
noticing that Eq. (I109p holds for both homoscedastic and heteroscedastic 
data. It can be seen that the lines of adjustment are oriented along the 
same direction (York, 1967) but are perpendicular to the regression line only 
in the special case, = 1, which is the genuine orthogonal regression (e.g., 
Carroll et al, 2006, Chap. 3, §3.4.2). 

Earlier formulations of the model with respect to the parent paper (Y66) 
may be found in several attempts (e.g., Kummell, 1879; Koopmans, 1937; 
Deming, 1943; Tintner, 1945; Lindley, 1947; Anderson, 1951; Madansky, 
1959) as well as later investigations (e.g., Barnett, 1967; Moran, 1971; Kendall 
and Stuart, 1979, Chap. 29; Fuller, 1980, 1987, Chap. 1, Sect. 1.3). 

Taking into due account Eqs. ([20]) and ffT09|l . Eqs. ([22]), ([M]), ([32]) and 
reduce to: 
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w 



(a^ + — 2racy ' 
rcWx 

(a^ + — 2rac)2 



1 < z < n 
1 < i < n 

1 < i < n 



* (a^ + — 2racY 
which, owing to Eq. (135 p . imphes the following: 

Qpq = kq{w^)pq ; Q = W,V,U,P 



(111) 
(112) 

(113) 
(114) 



where fcg = Qijiu^^ maintains constant. 

Accordingly, the pseudo cubic, Eq. and the pseudo quadratic, 

Eq. (1371) . reduce to: 



[rc(wx)2o - (wx)ii]a^ - [c^(wx)2o - (wx)o2]a - \rc{w^Q2 - c^(wxOii] = 0(115) 

where r = sgn(r) in the limit of completely correlated errors in X and in Y . 

In the special cases, (? — )■ +oo, (? — )■ 0, Eq. (11151) reduces to (167|) and (l92l) . 
respectively, as expected. The regression line slope and intercept estimators, 
a and 6, can be derived from Eqs. (11151) and (12B]) . respectively, where the 
parasite solution of the pseudo quadratic equation, in the former case, must 
be dismissed. 

With regard to the regression line slope and intercept variance estimators, 
((T^^)^ and [a'-^^'^ , with no account taken of the scatter of the data points, 
Pj = (Xj, "Kj), about the regression line, in the case under discussion Eqs. ( HUj) 
and fH3l) reduce to: 



dhc 



0- 



E 

1=1 



' d(j) 



2rc 



[a 



bc> 



E 



' dip 



dijj 



dtp dtp 
dXdY. 



dXi dY 



— ;(ii6) 



;ii7) 



and the substitution of Eqs. fl226|) . fl227D . and fl228D . into ffTTHj) and ffTTTD . 
respectively, yields cumbersome expressions of the regression line slope and 
intercept variance estimators, which shall not be explicitly written here. 

The squared residual trace, expressed by Eq. (12381) . in the case under 
consideration via Eqs. fl35|) and (IllOp reduces to: 



{acf{wx)2o + {wx)o2 - 2ac(wx)ii 
(ac)^ + — 2rcac 



:ii8) 
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and concerning the regression line slope and intercept variance estimators, 
(o"a(^)^ and {a-f^^y, with due account taken of the scatter of the data points, 
Pi = (Xj, Yi), about the regression hne, in the case under discussion Eqs. ( HT]) 
and fH5|) . by use of flllSp may be cast into a more explicit form. 

In the special case of uncorrelated errors, r — )■ 0, Eq. f lllSp reduces to: 



(w^)iia^ + [c^(wx)20 - (wx)o2]a - c^(wz)ii = 
which has the solutions (Deming, 1943): 



:ii9) 



ac = TTT^T <^ 1 T 

2(w^)ii 



1 + c- 



2 / [1^x)02 



20 



2(w^-)ii 



1/2- 



(120) 



and the regression line slope estimator is obtained disregarding the parasite 
solution. Then the substitution of Eq. f ll20p into f l2B]) yields the regression 
line intercept estimator, as: 



bc = Y- acX 



:i2i] 



where the index, C, denotes the case under discussion, with normal residuals 
(FB92). 

The squared regression line slope estimator, via Eq. (11200 . reads: 



(«c)' 



[Wx 02 - C^[Wx 20 



11 



]_ / {Wx)o2 - C^(Wx)2o ' 



C 



2(ifx)ii 



1/2 ^ 



(122) 



where the square root, if sufficiently close to unity, may be developed in 
binomial series and the terms of higher order neglected. The result is: 



2(w^-)ii 



2cA 2(w^)u 



{Wx)o2 - C^{Wx)20 

2{wx)n 



< 1 



123a) 



(123b) 



which, performing some algebra and neglecting the terms of higher order, 
takes the expression: 



(ac)' = c't 2c 



,W^j02 - C^yWx) 20 



1 / {w^)o2 - C^{Wx)2oV 



2(Wx)ll 



' 9 



2(w. 



< 1 ; (124) 



x)ll 
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and additional algebra shows that the following relation holds: 

{acf - {w^)02/{Wx)20 . , l(^^)ll 1 f{Wx)02-c'^{Wx)20y^^^ 

-l±-jT^^; ^ ^7^^^ <1;(125) 



where further attention has to be devoted to the special case: 



(ac)^ - (ti'x)02/(Wx)20 



1 ± A^^ = 1 - sgn[(w^)ii]A^^ ; (126) 



expressed in terms of the regression line correlation coefficient, A^^. 

With regard to the regression line slope and intercept variance estimators, 
{a'^J^ and ((T^^)^, the substitution of Eqs. f l229|) . fl233|) . and (EHD into ffTTHj) 
and flll7p particularized to uncorrelated errors, r = 0, yields after some 
algebra: 



/"/ \2 \2\"^xjuz I v^xyzu /I <->o\ 



where Eq. fl5^ has also been used. 

The squared residual trace, expressed by Eq. (11181) . in the case under 
consideration reduces to: 



^ iacf{wx)2o + {w^)o2 - 2ac(w^)ii ^ (ac)^(w:,.)2o - (^^x)o2 
^" (ac)2 + c2 " (ac)2-c2 '^'''''^ 



where Eq. (I232p has also been used. Concerning the regression line slope and 
intercept variance estimators, ((3"ac)^ and (o"g^)^, Eqs. fj^Tl) and fH5|) . by use 
of ([nSD, dM and (dSnD, after a lot of algebra reduce to: 

x2 _ 1 (Qc)^ [(^^)o2 + C^(w;)2o][(Qc)^(w;)20 - (w;)02] 

^""'^^ "n-2(ac)^-c2 [(5r.)n]2 ' ^ ' 



1 1 (ac)2 + c2 



n - 2 (m,.)oo (ac)2 - c2 



(ac)'(tir,)2o - (^r,)o2 +(X)2(<Tac)' ; (132) 



where further attention has to be devoted to the special case, c — )■ Sc. 

In the limit of errors in X negligible with respect to errors in y, c — )■ +00, 
Wxi/c^ — ^ Wy^, dc — )■ ciY, it can be seen that Eqs. (11311) and (I132p reduce to 
S]) and fl76l) . respectively, as expected. 
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In the limit of errors in Y negligible with respect to errors in X, c — )■ 0, 
c^Wy. — )■ Wx^, cic o,x, it can be seen that Eqs. fll3ip and fll32p reduce to 
fllOOp and f llOip . respectively, as expected. 

In the limit of the special case considered above, — )■ (u^) 02/ ("5^) 20, the 
combination of Eqs. f ll26p and f ll32p yields: 



. [(«c)' + c']{l - sgn[(e;.)ii]A^J + {Xfiaa^y ; (133) 

n — z {Wx)oo 

in terms of the regression line correlation coefficient, A^^,^. 

The parameter, c^, appearing in Eqs. fll3ip and (11321) . may be eliminated 
via Eq. fl232|) . The result is: 



2 ^ [t^x 02 - a'C[u^x 11 
c = ac . 



(ac)^ - 



ac[{acY{wl)2o - {wx)o2] 



(134) 
(135) 

ac[Wx)20 - [Wx)ll 

and the substitution of Eqs. (I134p and (11351) into (I13ip yields after some 
algebra: 



[CT. 



n-2\ [{w,)iiY 



+ 



D, 



+ 2 



[Wx)o2 + [ac] {.Wx)20 



(136) 



where D^^ is a weighted deviation determinant, Eq. (I103p . 

In the special case of homoscedastic data, w^. = w^, 1 < i < n, which 
implies Qi = Q = W,V,U,P, via Eqs. ([IIU])-([n3D, and Eqs. and ^ 
hold. Accordingly, Eqs. ffT20|l . ffmjl . fflMD . ffT32|) . flT33l) and ffM reduce to: 



5', 



ac 



02 



20 



2^11 



It 



1 + c^ 



'5, 



02 



c^S- 



-2 



20 



1/2- 



hc = Y- dcX 
.0 1 



[CT. 



2Sn ; 
' [So2 + c^S2o][{ac)'S2o-S, 



02J 



n - 2 (ac)2 - c2 
1 1 (ac)' + c2 



6c ^ 



n — 2n (o-c)' ~ 
1 1 



{ac fS2o - S02 + (X)2(aae) 



2 . 



[a. 



n — 2n 



^[(ac)' + c'][l - sgn(Sn)As] + iX)\a,^y ; 



n 



(^11)' 



(ac)'5'2o 



,(^11)' 



dcS_ 



11 



(137) 

(138) 
(139) 

(140) 
(141) 
(142) 
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where is the deviation determinant, Eq. (I57|) . and 

= re: Q 11/2 5 c = ^ ! (143) 

P20'-'02j ' '-'20 

is the regression hne correlation coefficient, and the above value of the pa- 
rameter, c^, relates to Eq. fll4ip . 

The expression of the regression line slope estimator, Eq. fll37p . coincides 
with its counterpart determined using the method of moments estimators 
[e.g.. Fuller, 1987, Chap. 1, §1.3.2, Eq. (1.3.7) therein]. More specifically, the 
method of moments estimators and the least square estimators of ac are the 
same (e.g.. Fuller, 1987, Chap. 1, §1.3.3). 

The expression of the regression line slope variance estimator, Eq. (11421) . 
is different from its counterpart calculated in an earlier attempt (FB92), due 
to a different term within round brackets on the right-hand side of Eq. (11421) . 
For a formal demonstration, see Appendix [Dl After long and strong work, 
it can be seen that the expression of both the regression line slope variance 
estimator determined in an earlier attempt (FB92) and intercept variance 
estimator expressed by Eq. (11401) . coincide with their counterparts reported in 
specific textbooks for structural models and uncorrelated errors (e.g.. Fuller, 
1987, Chap. 1, Sect. 1.3). 

In the special 1, the above results reduce to their counterparts 

related to genuine orthogonal regression, where the lines of adjustment are 
perpendicular to the regression line (Adcock, 1877, 1878; Pearson, 1901; 
Jones, 1937; Teissier, 1948; Kermack and Haldane, 1950). 

Turning back to the general case of weighted residuals, but restricting to 
the special case, = {w'x)o2/{wx)2o, the pseudo quadratic, Eq. (11191) . reduces 
to: 

- = ; (144) 
which has the solutions (Kermack and Haldane, 1950; Y66): 



Or = =F 



{Wx)20 



1/2 

; (145) 



and the regression line slope estimator is obtained disregarding the parasite 
solution. Then the substitution of Eq. (I145p into yields the regression 
line intercept estimator, as: 

bK = Y-anX ; (146) 

where the index, R, denotes the case under discussion, with normal residuals 
(Y66). 
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The regression line slope and intercept variance estimators are obtained 
by substitution of Eq. fll45p into fll3ip , (11331) and (11361) . After some algebra, 
the result is: 



[cr, 



n-2 (w^) 



20 



[at 



-^^{1 - sgn[(^r.)n]A^J + {XYia,,: 



[cr. 



[aR) 



n-2 [[{w^)n]'' 



+ 



+ 2 -2a 



11 



(147) 
(148) 
(149) 



where is a weighted deviation determinant, Eq. (11031) . 

In the special case of homoscedastic data, w^^ = w^, 1 < i < n, which 
implies Qi = Q,Q = W, V, U, P, via Eqs. ffTT0D- ffTT3|) . and Eqs. ([79]) and ([80]) 
hold. Accordingly, Eqs. (I145p - (I149I) reduce to: 



Or = =F 



r. . 1/2 

bTx = Y - a^X ; 
. . X o 2 5*02 

n - 2 

2 '5'o2 
n — 2 n 



cr, 



[l-sgn(^n)As] + (X)2(aaJ' 



(or)^ 



n 



+ 



'11 



+ 2 - 2aR— 

iir "Jii 



(^:-^2 



(150) 
(151) 
(152) 

(153) 

(154) 



where Ds is the deviation determinant, Eq. f[87]) . 

The expression of the regression line slope variance estimator, Eq. (I154p . 
is different from its counterpart calculated in an earlier attempt (FB92) , due 
to a different term within square brackets on the right-hand side of Eq. (I154p . 
For a formal demonstration, see Appendix [Dl 



2.8 Extension to structural models 

A nontrivial question is to what extent the above results, valid for functional 
models, can be extended to structural models. In general, assumptions re- 
lated to structural models are different from their counterparts related to 
functional models (e.g., Buonaccorsi, 2006; 2010, Chap. 6, §6.4.5) but, on the 
other hand, they could coincide for a special subclass. In any case, what- 
ever different assumptions and models can be made with regard to structural 
and functional models, results from the former are expected to tend to their 
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counterparts from the latter when the intrinsic scatter is neghgible with re- 
spect to the instrumental scatter. It is worth noticing that most work on 
linear regression by astronomers involves the situation where both intrinsic 
scatter and heteroscedastic data are present (e.g., AB96; Tremaine et al., 
2002; Kelly, 2007). 

In structural models, a true point, P*{x*,y*), lying on the (true) regres- 
sion line, is shifted by intrinsic scatter to an actual point, Pg-(xsi, ysi), outside 
the regression line. This last, in turn, is shifted by instrumental scatter to 
an observed point, Pi{Xi,Yi). 

The coordinates of observed and actual points, according to Eq. ([2]), and 
the coordinates of actual and true points, are assumed to be related as: 

(^fJ. = ^.-^Si ; Z = X,Y ; l<z<n ; (155) 
(^sji = zsi- z* ; z = x,y; l<i<n ; (156) 

where the random variables, (^fJj, (^sji, obey the distributions, /F2i[(^Fji], 
/s^j[(^Sz)t], respectively. While the former distribution is necessarily Gaus- 
sian, depending only on mesaurements processes, the latter distribution may 
be different, depending on a larger variety of processes. 

With regard to true points, the substitution of Eq. (11561) into ([T]) yields: 

y: + {^Sy)i = a[x* + {^sM + b + ei ; (157) 

which, owing to Eq. (jl]), reduces to: 

e^ = {^Sy)^ - ai^sji ; (158) 

where the contribution of each variable to the intrinsic scatter is explicitly 
expressed. For the true regression line i.e. fixed slope, a, a null expectation 
value of ej necessarily implies null expectation values of (^s^)i and (^s^)*) 
1 < i < n, and vice versa. 

With regard to observed points, the substitution of Eq. (11551) into ([T]) 
yields: 

- (^fJ. = a[X, - (^fJ.] + b + e, ; (159) 
which, using Eq. (11581) . takes the form: 

Y, = aX, + b+ [{^sy)^ + {^fM - ali^sji + {^fM ; (160) 

where the contribution of each variable to the intrinsic scatter is explicitly 
expressed. 

A special subclass of structural models is defined according to the follow- 
ing assumptions. 
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(a) The random variables, (CF^)i, (Cs^)i) z — x,y, 1 < i < n, are indepen- 

dent. 

(b) The distributions, fs^i[{Csz)i]j ^ = x,y, ^ ^ i ^ n^, related to the intrinsic 

scatter, are Gaussian and corresponding expectation values are null. 

(c) The sum of variances related to the distributions, /F^i[(CF^)i], fszi[{isz)i]j 

z = x,y, I < i < n, maintains constant as: 

[Mi? + [Mi? = [M? = (const^)^; z^x,y; 1 < i < n. (161) 
Owing to assumption (b), the related distributions read: 

V27r((7uji [ 2[(cTuJiJ^J 
U = F,S; z^x,y ; 1 < i < n ; (162) 

with regard to the random variables, (^u^)i- 
The random variables: 

{Cz)i = Mi+M^ = Zi-z*; Z = X,Y; z = x,y; l<t<n; (163) 

via assumption (a) obey the distribution: 

UiLU = fFAi^FMfsAi^sM ; (i64) 

which, using a theorem of statistics, via assumptions (a), (b), is also Gaus- 
sian, expressed as: 

[{'yz)i?^[Mi? + [Mi? ; z^x,y ; l<i<n ; (166) 

accordingly, residuals obey a Gaussian distribution and the weights, w^;., Wy^, 
1 < i < n, remain unchanged via assumption (c). Then, for a selected regres- 
sion estimator, the regression line slope and intercept variance estimators are 
independent of the amount of instrumental and intrinsic scatter, including 
the limit of null intrinsic scatter (functional models) and null instrumen- 
tal scatter (extreme structural models). In this view, the whole subclass of 
structural models under consideration could be related to functional mod- 
elling (Carroll et al, 2006, Chap. 2, §2.1). 
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3 An example of astronomical application 



3.1 Astronomical introduction 

With regard to stellar populations, the dependence of oxygen abundance 
on iron abundance, or [0/H]-[Fe/H] relatioru, has been deeply investigated 
during the last decade (e.g., Carretta et al., 2000; Israelian et al., 2001a,b; 
Barbuy et al. (eds.), 2001; Jonsell et al., 2005; Fulbright et al., 2005; Garcia 
Perez et al., 2006; Melendez et al., 2006; Fabbianet al., 2009, hereafter quoted 
as Fal09; Rich and Boesgaard, 2009, hereafter quoted as RB09; Schmidt et 
al., 2009, hereafter quoted as Sal09). 

It has been realized that the [0/H]-[Fe/H] relation is strongly dependent 
on both the selection of the spectroscopic oxygen lines and the choice of 
the atmosphere model. The discrepancy due to using different methods and 
different models remains large, and no general consensus on the best choice 
still exists. For further details refer to an earlier attempt (Caimmi, 2010). 

Oxygen is the most abundant metaH in the universe, but it is more dif- 
ficult than iron to detect. The population of available samples where oxy- 
gen abundances are directly determined, does not exceed a few hundreds at 
most (e.g., Ramirez et al., 2007; Melendez et al., 2008; RB09; Fal09; Sal09). 
Oxygen abundance in larger samples may be deduced by use of an inferred 
[0/H]-[Fe/H] relation. 

According to the stellar evolution theory, oxygen is produced only via 
type II supernovae (SnII), characterized by massive (m ~ Sm©) progenitors. 
On the contrary, iron is produced also via type la supernovae (Snia), where 
a white dwarf (m < mc) attains the Chandrasekhar limit (mc ~ 1.4m0) due 
to mass accretion from a close red giant companion, where white dwarfs are 
related to low-mass (m ~ Sm©) progenitors. During the lifetime of primeval 
SnII progenitors, r ~ O.lGyr, a linear [0/H]-[Fe/H] relation is expected, 
while the contrary holds at later times unless SnII contribution to iron pro- 
duction remains dominant. Three samples shall be used for bivariate least 
squares linear regression, namely RB09, Fal09, Sal09, where the denomina- 
tion comes from related parent papers. 

The RB09 sample (A^ = 49) is made of a homogeneous subsample (A^ = 
24) of metal-poor (—3.5 < [Fe/H] < —2.2) stars, and a non homogeneous sub- 

^For a generic nuclide, N, the logarithmic number abundance is defined as [N/H] = 
log(N/H) — log(N/H)0, normalized to hydrogen, H, and to solar abundance, (N/H)©. The 
related mass abundance is defined as (j)^ — Z^/{Z^)q. The relation: log(/)N = [N/H] 
holds to a good extent (Pagel, 1989; Malinie et al., 1993; Rocha-Pinto and Maciel, 1996; 
Caimmi, 2007). 

^In astrophysical language, all elements heavier than helium are called "metals" . 
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sample {N = 25) of higher-metallicity (—3.1 < [Fe/H] < —0.5) stars. In both 
cases, the stellar population remains unspecified and oxygen abundance has 
been determined using standard local thermodynamical equilibrium (LTE) 
one- dimensional hydrostatic model atmospheres. Standard deviations are 
provided for each star, where typical values are o"[Fe/H] = "^[o/h] = 0.15. For 
further details refer to the parent paper (RB09). 

The Fal09 sample (A^ = 44) is made of halo stars (-3.3 < [Fe/H] < 
—1.0) where oxygen abundance has been determined using three different 
methods involving (a) LTE one-dimensional hydrostatic model atmospheres; 

(b) three-dimensional hydrostatic model atmospheres in absence of LTE with 
no account taken of the inelastic collisions via neutral H atoms (Sh = 0); 

(c) three-dimensional hydrostatic model atmospheres in absence of LTE with 
due account taken of the inelastic collisions via neutral H atoms (Sh = 1). 
Standard deviations are not reported for each star, but typical values are 
mentioned to be o"[Fe/H] = (^[o/h] = 0.15. For further details refer to the 
parent paper (Fal09). 

The RB09 and Fal09 samples have iV = 11 (necessarily halo) stars in com- 
mon, where the values assumed for effective temperature and surface gravity 
have been determined using different methods, yielding different values for 
each star. For further details refer to an earlier attempt (Caimmi, 2010). 

The Sal09 sample (A^ = 63) is made of cool (late K and M) dwarfs (—1.8 < 
[Fe/H] < +0.2) where oxygen abundance has been determined by use of the 
7 i?2 — TiO band at 7054 A combined with previously derived abundances 
of Ti and Fe. Standard deviations are provided for each star, where typical 
values may be estimated as crpe/H] = "^[o/h] = 0.15. A single star, LHS 185, is 
mentioned (Sal09, Table 1 therein) but excluded from further analysis. For 
further details refer to the parent paper (Sal09). 

In conclusion, the Fal09 sample is made of homoscedastic data, while 
the remaining RB09 and Sal09 samples are made of heteroscedastic data. 
To a first extent, the latter may be considered as made of homoscedastic 
data where standard deviations are approximated to related typical values. 
Under the further assumption that intrinsic scatter is negligible with respect 
to instrumental scatter, i.e. functional models, the general results of section 
[2] may be particularized to the case under discussion, where errors in [Fe/H] 
and [0/H] may be considered as uncorrelated (r^ = 0, 1 < i < n) to a good 
extent. 

3.2 Statistical results 

The [0/H]- [Fe/H] empirical relations are interpolated using the regression 
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Table 1: Regression line slope and intercept estimators, a and b, and related 
dispersion estimators, aa, and a^, for heteroscedastic models, G, Y, X, O, R, 
applied to the [0/H]-[Fe/H] empirical relation deduced from the following 
samples (from up to down): RB09, Sal09. Values related to different slope 
and intercept dispersion estimators (Y66) are also reported for comparison 
with current results (GRS). For G models, slope and intercept dispersion 
estimators were not evaluated in the present attempt. For Y models, different 
slope or intercept dispersion estimators yield coinciding values, as expected. 



m 


a 




i 


b 


a 


b 


sample 






GRS 


Y66 




GRS 


Y66 




G 


0.7279 




0.0294 


+0.0043 




0.0672 


RB09 


Y 


0.6714 


0.0314 


0.0314 


-0.1121 


0.0675 


0.0675 




X 


0.7305 


0.0290 


0.0279 


+0.0316 


0.0735 


0.0712 







0.6964 


0.0278 


0.0271 


-0.0512 


0.0707 


0.0689 




R 


0.7050 


0.0282 


0.0272 


-0.0305 


0.0725 


0.0693 




G 


0.6383 




0.0435 


+0.0619 




0.0251 


Sal09 


Y 


0.6167 


0.0398 


0.0398 


+0.0439 


0.0198 


0.0198 




X 


0.8652 


0.0829 


0.0664 


+0.3080 


0.0673 


0.0575 







0.6355 


0.0637 


0.0541 


+0.1461 


0.0525 


0.0469 




R 


0.6927 


0.0700 


0.0560 


+0.1864 


0.0549 


0.0485 





models, G, Y, X, O, R, for heteroscedastic data (RB09 and Sal09 samples) 
and Y, X, O, R, for homoscedastic data (Fal09 sample, cases LTE, SHO, SHI) 
and heteroscedastic data where instrumental scatters are taken equal to re- 
lated typical values, crpe/H] = 0.15, ctjo/h] = 0.15, for both FB09 and Sal09 
samples. Slope and intercept estimators together with related dispersion es- 
timators are listed in Tables [1] and [2] for heteroscedastic and homoscedastic 
data, respectively. Also listed are values of slope and intercept dispersion 
estimators by earlier attempts (Y66; FB92) for comparison with their coun- 
terparts calculated in the current paper (GRS). 

Owing to high difficulties intrinsic to the determination of slope and in- 
tercept dispersion estimators in the general case, related calculations were 
not performed in dealing with G models and only approximate expressions 
(Y66), Eqs. ( HHj) and P9|) . were used. The regression line slope and in- 
tercept estimators are calculated using Eqs. and respectively. For 
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Table 2: Regression line slope and intercept estimators, a and b, and related 
dispersion estimators, (Xa, and ay for homoscedastic models, Y, X, O, R, 
applied to the [0/H]-[Fe/H] empirical relation deduced from the following 
samples (from up to down): RB09, Sal09, Fal09, cases LTE, SHO, SHI. 
Values related to different slope and intercept dispersion estimators (Y66, 
FB92) are also reported for comparison with current results (CRS). For Y 
models, different slope or intercept dispersion estimators yield coinciding 
values, as expected. 



ma Ga b ay sample 

CRS FB92 Y66 CRS Y66 



Y 


0, 


.6917 


0, 


.0317 


0, 


.0317 


0, 


.0317 


— 0, 


.0766 


0. 


,0737 


0. 


,0737 


FB09 


A 


U, 


. /oUU 


A 
U. 


nQ A Q 

,Uo4o 


U. 


,Uo4y 


U, 


nQQO 

.Uooz 


+u. 


,U 1 4z 


A 
U. 


,UoUo 


A 
U. 


,U M o 







0, 


.7143 


0. 


,0331 


0. 


,0327 


0, 


.0319 


-0, 


.0268 


0. 


,0766 


0. 


,0741 




R 


0, 


.7251 


0, 


.0336 


0, 


.0332 


0, 


.0321 


-0, 


.0030 


0, 


.0778 


0, 


.0746 




Y 


0, 


.5868 


0, 


.0461 


0, 


.0461 


0, 


.0461 


+0, 


.0908 


0, 


.0338 


0, 


.0338 


Sal09 


X 


0, 


.8077 


0, 


.0635 


0, 


.0637 


0, 


.0541 


+0, 


.2011 


0, 


.0430 


0, 


.0397 







0, 


.6476 


0. 


,0526 


0. 


,0509 


0, 


.0468 


+0. 


,1212 


0. 


,0363 


0. 


,0343 




R 


0, 


.6885 


0, 


.0562 


0, 


.0541 


0, 


.0479 


+0, 


.1416 


0, 


.0381 


0, 


.0352 




Y 


0, 


.8961 


0, 


.0303 


0, 


.0303 


0, 


.0303 


+0, 


.5476 


0, 


.0663 


0, 


.0663 


Fal09 


X 


0, 


.9381 


0, 


.0317 


0, 


.0318 


0, 


.0310 


+0, 


.6366 


0, 


.0693 


0, 


.0678 


(LTE) 





0, 


.9150 


0, 


,0311 


0. 


,0310 


0, 


.0305 


+0, 


,5877 


0, 


,0680 


0, 


,0666 




R 


0, 


.9168 


0, 


.0312 


0, 


.0310 


0, 


.0305 


+0, 


.5916 


0, 


.0681 


0, 


.0667 




Y 


1, 


.2261 


0, 


.0432 


0, 


.0432 


0, 


.0432 


+0, 


.8717 


0, 


.0945 


0, 


.0945 


Fal09 


X 


1, 


.2884 


0, 


.0454 


0, 


.0454 


0, 


.0443 


+1, 


.0037 


0, 


.0991 


0, 


.0968 


(SHO) 





1, 


.2640 


0. 


,0448 


0. 


,0445 


0, 


.0436 


+0. 


,9519 


0. 


,0978 


0. 


,0953 




R 


L 


.2569 


0, 


.0445 


0, 


.0443 


0, 


.0435 


+0, 


.9369 


0, 


.0973 


0, 


.0950 




Y 


1, 


.0492 


0, 


.0341 


0, 


.0341 


0, 


.0341 


+0, 


.6518 


0, 


.0745 


0, 


.0745 


Fal09 


X 


1, 


.0946 


0, 


.0356 


0, 


.0356 


0, 


.0348 


+0, 


.7479 


0, 


.0777 


0, 


.0761 


(SHI) 





1, 


.0732 


0. 


,0350 


0. 


,0349 


0, 


.0343 


+0. 


,7027 


0. 


,0765 


0. 


,0750 




R 


1, 


.0716 


0, 


.0350 


0, 


.0348 


0, 


.0343 


+0, 


.6993 


0, 


.0764 


0, 


.0750 
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the remaining models, the regression hne slope and intercept estimators and 
related dispersion estimators are calculated using Eqs. (jHSD, fl^ - fl75]) . fl7^ . 
and (l82i) . (183|) . (18^ . (185|) . case Y, heteroscedastic and homoscedastic data, 
respectively; Eqs. (ES]), ([94]), ([lOO]), ([lOl]), and ([IDID, ([l05]), (II06]), flTOTll . 
case X, heteroscedastic and homoscedastic data, respectively; Eqs. (11201) . 
dMD, (HMD, dM, and (inZD, (IISSl), (USHl), dHO]), all related to the spe- 
cial value, = 1 (genuine orthogonal regression), case O, heteroscedastic 
and homoscedastic data, respectively; Eqs. (I145p . (I146p . (I147p . (I148p . and 
(11501) . (I15ip . (11521) . (I153p . case R, heteroscedastic and homoscedastic data, 
respectively. 

The regression lines determined by use of the above mentioned methods 
are plotted in Figs. [T] and [2] for heteroscedastic and homoscedastic data, re- 
spectively, where sample denomination and population are indicated on each 
panel together with model captions. Homoscedastic data are conceived as 
a special case of heteroscedastic data in Fig.[I] to test the computer code, 
which is different for heteroscedastic and homoscedastic data. It can be seen 
that lower panels of Figs.[T] and [2] coincide, and the regression lines related to 
models G and O in lower panels of Figs. [T] and [2] also coincide, as expected. 
The whole set of regression lines for all methods and all samples is shown in 
the upper right panel of Figs. [1] and [2j 

An inspection of Tables [Tll2] and Figs. [Tll2] discloses the following. 

(1) Either of the inequalities (Ial90): 



holds for both heteroscedastic and homoscedastic data. In addition, 
dy < ciQ < dx for heteroscedastic data, but a counterexample is pro- 
vided in an earlier attempt (Y66). 

(2) Slope and intercept estimators by different methods are consistent within 

=F(T for samples with lower dispersion (FB09, Fal09), while the contrary 
holds for samples with higher dispersion (Sal09), with regard to both 
heteroscedastic and homoscedastic data. 

(3) Slope and intercept dispersion estimators coincide with their counter- 

parts related to earlier attempts for both heteroscedastic (Y66) and 
homoscedastic (Y66; FB92) data, in the special case of Y models. For 
the other models, the approximations exploited in an earlier attempt 
(Y66) make lower limits with respect to current results, while an al- 
ternative expression of the slope dispersion estimator (FB92) yields 
slightly different results. 



fly < < 0.R < 1 < 

o-Y < 1 < Or < flo < Q-x 



^11 > 
Sn > 



(167a) 
(167b) 
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(4) Systematic variations due to different sample data are dominant with 
respect to the instrumental scatter. 

In conclusion, regression lines deduced from different sample data represent 
correct (from the standpoint of regression models considered in the current 
attempt) [0/H]-[Fe/H] relations, but no definitive choice can be made until 
systematic errors due to different methods and/or spectral lines in determin- 
ing oxygen abundance, are alleviated. 

4 Discussion 

For an assigned sample, structural models belonging to a special sub- 
class are indistinguishable from functional models provided restrictions (a)- 
(c) hold as outlined in subsection 12.81 Accordingly, the results of the current 
paper also apply to structural models of the kind considered. The expres- 
sion of regression line slope and intercept estimators and related variance 
estimators in terms of weighted deviation traces, for heteroscedastic and ho- 
moscedastic data, makes a first step towards a unified formalism of bivariate 
least squares linear regression. The bisector method has not been dealt with 
in earlier attempts related to functional models (Y66; Y69), but only in later 
investigations involving extreme structural models (Ial90; FB92). For this 
reason, the bisector method has not been considered in the current paper. 

Exact expressions of regression line slope and intercept estimators and re- 
lated variance estimators have been determined from general formulae (Y69) 
in the limit of generalized orthogonal regression i.e. {o-yy)i/ {axx)i = c^, 
1 < i < n. It is noteworthy that a constant variance ratio, c^, for all data 
points, does not necessarily imply equal variances, {<Jxx)i = o'xx = const, 
{cryy)i = CTyy = coust, 1 < i < Ti. Whllc regression line slope and intercept 
estimators attain a coinciding expression in different attempts (Y66; Y69; 
Ial90; FB92) with regard to a fixed model, the results of the current paper 
show that the contrary holds for related variance estimators. 

Approximate expressions provided in earlier attempts (Y66; Y69) make 
(at least in computed cases) a lower limit to their exact counterparts, as 
shown in Tables [T]121 On the other hand, alternative expressions given in 
a later investigation, restricted to regression line slope variance estimators, 
yield different results (FB92). 

The above mentioned discrepancy could be explained in different ways, 
namely: (1) calculations performed in the current paper were checked and 
repeated twice or more, but something wrong cannot be excluded; (2) the 
expression of the regression line slope variance estimator determined in the 
current paper, is approximate intead of exact, contrary to what reported in 
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the parent paper (Y69); (3) the expression of the regression hne slope variance 
estimator used in an earher attempt [FB92, Eq. (14) therein] for structural 
models with normal residuals and dominant intrinsic scatter, is approximate 
instead of exact; (4) for generalized orthogonal regression, the expression of 
the regression line slope variance estimator is different for functional (Y69) 
and structural (FB92) models even if, in the latter case, residuals obey a 
Gaussian distribution and the intrinsic scatter is dominant with respect to 
the instrumental scatter; (5) with regard to the regression line slope variance 
estimator, the method of partial differentiation, used in an earlier attempt 
(Y69) and in the current paper, yields different results with respect to the 
method of moments estimators [e.g.. Fuller, 1987, Chap. 1, §1.3.2, Eq. (1.3.7) 
therein] . 

It is well known that the regression line slope and intercept estimators 
for Y models, Eqs. (182]) and (183|) . are biased (e.g.. Fuller, 1987, Chap. 1, 
§1.1.1; Carroll et al., 2006, Chap. 3, §3.2; Buonaccorsi, 2010, Chap. 4, §4.4). 
Biases can be explicitly expressed in the special case of homoscedastic models 
where random variables obey Gaussian distributions. More specifically, the 
condition 1 — p2o ^ 1 ensures bias effects are negligible, where p2o is the 
reliability ratio: 

P20 = ^ — , . ^° ^^ ; (168) 

which implies < p2o ^ 1- For further details refer to specific monogra- 
phies (e.g.. Fuller, 1987, Chap. 1, §1.1.1; Carroll et al, 2006, Chap. 3, §3.2.1; 
Buonaccorsi, 2010, Chap. 4, §4.4). Accordingly, the reliability ratio cannot 
exceed unity, or in other words the regression line slope estimator, Eq. (1821) . 
is biased towards zero, as clearly shown in current literature (e.g., Kelly, 
2007). Following a similar line of thought with regard to the regression line 
slope estimator for X models, Eq. (11041) . discloses the last is biased towards 
infinity, in the sense that the true slope is overestimated. 

The regression line slope estimator for O models (genuine orthogonal re- 
gression) and R models (major-axis regression) lie between their counterparts 
related to Y and X models, according to Eq. fll67p . which implies bias correc- 
tions (e.g., Carroll et al., 2006, Chap. 3, §3.4.2). Though there is skepticism 
about an indiscriminate use of generalized orthogonal regression estimators, 
still it is accepted the method is viable provided both instrumental and intrin- 
sic scatter are known (e.g., Carroll et al., 2006, Chap. 3, §3.4.2; Buonaccorsi, 
2010, Chap. 4, §4.5). 

With regard to heteroscedastic data, an inspection of Tables [T]l2] shows 
that for lower data dispersion (RB09 sample) the values of regression line 
slope and intercept estimators, deduced for heteroscedastic (Table [1]) and 
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homoscedastic (Table [2]) data, are systematically smaller in the former case 
with respect to the latter, but remain consistent within =po". For larger disper- 
sion data (Sal09 sample) no systematic trend of the kind considered appears, 
but the values of regression line slope and intercept estimators are still con- 
sistent within =p(T in different alternatives. It may be a general property of 
the regression models considered in the current attempt or, more realistically, 
intrinsic to the samples selected for the application performed in section [31 

The reliability ratio, Eq. fll68p . has been calculated for all sample data 
together with its counterpart for X models: 



and the inequalities, p2o > 0.92, po2 > 0.91, hold in any case except po2 > 0.86 
for the Sal09 sample, which implies poorly biased regression line slope and 
intercept estimators for the samples considered using Y and X models and, 
a fortiori, using O and R models. 

5 Conclusion 

From the standpoint of a unified analytic formalism of bivariate least 
squares linear regression, functional models have been conceived as structural 
models where the intrinsic scatter is negligible (ideally null) with respect to 
the instrumental scatter. 

Within the framework of classical error models, the dependent variable 
has been related to the independent variable according to the well known 
additive model (e.g., Carroll et al., 2006, Chap. 1, §1.2, Chap. 3, §3.2.1; 
Buonaccorsi, 2010, Chap. 4, §4.3). Then the classical approach pursued in 
earlier papers (Y66; Y69) has been reviewed using a new formalism in terms 
of weighted deviation traces which, for homoscedastic data, reduce to usual 
quantities, leaving aside an unessential (but dimensional) multiplicative fac- 
tor. 

Regression line slope and intercept estimators, and related variance esti- 
mators, have been expressed in the general case of correlated errors in X and 
in Y for heteroscedastic data, and in the opposite limiting situations of (i) 
uncorrelated errors in X and in Y , and (ii) completely correlated errors in X 
and in Y . The special case of (C) generalized orthogonal regression has been 
considered in detail together with well known subcases, namely: (Y) errors 
in X negligible (ideally null) with respect to errors in Y\ (X) errors in Y 
negligible (ideally null) with respect to errors in X; (O) genuine orthogonal 
regression; (R) reduced major-axis regression. 
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In the limit of homoscedastic data, the results determined for functional 
models have been compared with their counterparts related to extreme struc- 
tural models i.e. the instrumental scatter is negligible (ideally null) with 
respect to the intrinsic scatter (Ial90; FB92). While regression line slope 
and intercept estimators for functional and structural models have necessar- 
ily been found to coincide, the contrary has been shown for related variance 
estimators even if the residuals obey a Gaussian distribution, with the ex- 
ception of Y models. 

An example of astronomical application has been considered, concerning 
the [0/H]-[Fe/H] empirical relations deduced from five samples related to 
different stars and/or different methods of oxygen abundance determination. 
For selected samples and assigned methods, different regression models have 
been found to yield consistent results within the errors (=F0") for both het- 
eroscedastic and homoscedastic data. Conversely, it has been shown that 
samples related to different methods produce discrepant results, due to the 
presence of (still undetected) systematic errors, which implies no definitive 
statement can be made at present. A comparison has also been made be- 
tween different expressions of regression line slope and intercept variance 
estimators, where fractional discrepancies were found to be not exceeding a 
few percent, which grows up to about 20% in presence of large dispersion 
data. 

An extension of the results to structural models has been left to a forth- 
coming paper. 

6 Note added in proof 

The author is indebted with G.J. Babu and E.D. Feigelson for providing an 
earlier version of the erratum of their quoted paper (FB92) before publication 
(Feigelson and Babu, 2011). The quotation FB92 throughout the text has to 
be intended as including the original paper and the erratum (Feigelson and 
Babu, 1992, 2011). 

7 Addendum 

The slope variance estimator for generalized orthogonal regression with nor- 
mal residuals, expressed in an earlier attempt [FB92, Eq. (4) therein, here- 
after quoted as Eq. (FB4ev)] has been revised [FB92, erratum 2011, first 
equation therein, hereafter quoted as Eq. (FB41v)]. Accordingly, the deriva- 
tion of Eqs. fl2i2|) . f l25T|) . fl2H3|) . which started from Eq. (FB4ev), should be 
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repeated using Eq. (FB41v). It can be seen the latter is closer to Eq. f ll42p 
than the former. The difference is small for the numerical application shown 
in Sect.ini 
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Appendix 



A Solutions of cubic equations 

Without loss of generality, a cubic equation may be cast under the stan- 
dard form: 

+ aiz^ + a2-2 + as = ; (170) 
where the discriminant reads: 

D = Q^ + R^ ■ (171) 

Q = ^Y^ ; (172) 

9aia2 - 27a3 - 2a\ 

R= ^ ; (173) 

and one (the other two being complex coniugate) or three real solutions exist, 
according if D > ot D < 0, respectively. The related expressions are: 

z, = {R + D'^Y^ + {R- ^1/2)1/3 _ 1 ^ . (174) 

for D > and: 



zi = 2^Pq cos (^^ + - \<^i ; (175) 

'e 2ti\ 1 



Z2 = cos + Y J - 3«i ; (176) 

'e 47r\ 1 



2:3 = 2^-Qcos I 3 + Y 1 - 3«i ; (177) 



6 = arctan — - — ; (178) 
R 

for D < 0, where 6 has to be replaced by =F tt as appropriate, when a 
discontinuity of the arctan function occurs. A null factor has been put into 
Eq. fll75p to save aesthetics. 



B Determination of partial derivatives 

The explicit expressions of the regression line slope and intercept vari- 
ance estimators, imply the calculation of the partial derivatives appearing 
in Eqs. f HOj) and fH3|l . respectively. With regard to the regression line slope 
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variance estimator, (a^)^, the slope estimator, a, has to be considered as an 
independent variable in performing partial derivatives, d/dZi, Z = X,Y. 
Using Eqs. (l22i) . (125|) and (135|) . the partial derivatives of the deviation from 
the weighted mean, with respect to the coordinates, read: 

^ 'Ze-Z) = 6,e-W: ; (179) 



dZi 



W: = j^ ; Woo = Y.Wi = nW ; (180) 

1^00 i=l 

where 5ie is the Kronecker symbol and the prime does not mean first deriva- 
tion. 

Using Eqs. fl35l) and fll79p . the partial derivatives of the weighted deviation 
traces, with respect to the coordinates, read: 



^=pQ,{X,-Xy-\Y,-Yy-pW:Qp^i,, ■ p>0 ■ (181a) 
dQ 



pq 



dYi 



qQi{X,-XY{Y,-Yy'^ ~qW[Qp,^.^ ; g>0 ; (181b) 



§^^0; If.O; (1810 



where the cases of interest are Qpq = Vn, U20, P20, V02, Pu, Uq2- 

Using Eqs. fl22|) . fl3T]) . fl32|) and f l38|l . the partial derivatives of the weights, 
Wi^ with respect to the regression line slope estimator, a, read: 

dWi 

2{U,~aV,) ; (182) 



and, in addition: 



da 



dV- 

-Q^ = m-aVl) ; (183) 

VI ^ ^ ; (185) 

^=AiU:'-aU:) ; (186) 

u-=^ ; (187) 
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m 

da 
P' = 



_wl_ 

' 



(188) 
(189) 

(190) 



where the prime, the second, and the third, do not mean first, second, and 
third derivation, respectively. 

Using Eqs. fl35|) and fll83l) - fll90p . the partial derivatives of related weighted 
deviation traces, with respect to the regression line slope estimator, read: 



dVr 



pq 



da 



da 



pq 



da 



dR 



pq 



da 



A{UZ - aP' 



pqj 



(191) 
(192) 
(193) 



which, together with Eq. (11791) . complete the set of results needed for an ex- 
plicit expression of the partial derivatives of the function, 0(Xj, Fj, a), defined 
by Eq.dSH). 

Using Eqs. (15^ and (11811) . the explicit expression of the partial derivatives 
with respect to the coordinates, read: 



dX, 



{\y,{Y,-Y)-W%i 



+ 2 Pi{X,-X)-WlP, 



10 



dY, 



ViiX,-X)-WlV, 



10 



{[p,{x,-x)-w:p,o 



u,{x,-x)-w:uio 

'p,{Y,-Y)-W:Poi 
V,{Y, -Y)- W^Voi 
U,{Y, -Y)- Wfioi 



(194a) 



} ; (194b) 



and using Eqs. (l39i) . (11911) . (I192p . (I193p . the explicit expression of the partial 
derivative with respect to the regression line slope estimator reads: 



da 



- Ki)(«)' + 4(f/(i - f/^'o - P^o + K.2)(«) 
+ 2(2?7^'^ - 2?7^2 + 2P[^ - 21J'^^ + - V^^a 



+ (4[/^'2 - 4t/(;' + P20 - V02) ; 



(195) 
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finally, the substitution of Eqs. fll94p and (I195p into fllU]) yields an explicit 
expression of the regression line slope variance estimator, (o"^)^, with no 
account taken of the scatter of the data points, Pj = {Xi,Yi), about the 
regression line (Y69). 

With regard to the regression line intercept variance estimator, ((5"^)^, the 
regression line slope estimator, a, has to be considered as a function of the 
coordinates, a = Fa{Xi,Yi), 1 < i < n, in performing partial derivatives, 
d/dZi, Z = X,Y. Accordingly, dWi/dZi ^ 0. Using Eq. ([25]), the partial 
derivatives of the weighted mean, with respect to the coordinates, after some 
algebra read: 



dZ 



dZi w. 



00 L 



t~dZ~ 



X,Y 



(196) 



which, using Eqs. fl35]l and f ll82p . together with a theorem on the partial 
derivatives of the function of a function: 



dcp dcf) da 



dWf dWf da 



dZi dadZi ' dZi da dZi ' 
after additional algebra takes the explicit form: 

dX 1 



1 < i < n 



dXi 

dY 

dYi 



Wo, 
1 



W< 



00 



+ 2 (f/io - aVio 
+ 2 (f/c 



d<P/dXi 



aVr 



d(t)/da 
\ d<PldY, 



dcp/da 



(197) 

(198a) 
(198b) 



and using Eqs. (H2!) and (11980 . the exphcit expressions of the partial deriva- 
tives with respect to the coordinates read: 



dil) 
dX, 



dil) 
dY 



^dX ~da 

—a^rrz X 



dX, 
aW, 



W< 



00 



dX, 
2a 



dY 

dY, 



X 



w. 



+ 



00 



Woo 
da 
dY, 
2 



10 



X 



d(P/dX, 
d(j)/da 



(199a) 



W, 



00 



(f/oi 



-aVr 



01 



X 



d<p/dY, 
d4)/da 



(199b) 



and the substitution of Eqs. fll99p into (1430 yields an explicit expression of the 
regression line intercept variance estimator, {cry Y-, with no account taken of 
the scatter of the data points, Pj = (Xj, Fj), about the regression line (Y69). 
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In the special case of errors in X negligible with respect to errors in Y, 
analysed in subsection [531 Eqs. (dM]), (HMD, (HIS]), (HSZ]), (HSHD, (HHO]), via 
Eqs. fl63|) - (l66l) reduce to: 



w _ 

u: = o ■ 
v: = o ■ 

ur = o ; 
p.' = o ■ 



nw. 



y ' 



2=1 



(200) 

(201) 
(202) 
(203) 
(204) 
(205) 



and the particularization of Eq. ( 15^ to the case under discussion via Eqs. (|^ - 
fl66|) . selecting Q = V,U, P; p = 0,1; q = I, 0; and using Eq. fl25l) . yields: 



Qio = 
Qoi =0 



(206) 
(207) 



finally, the particularization of Eqs. fl5^ . fll94p . fll95p . ( 11991) . to the case 
under discussion via Eqs. flMl) - fpBl) and (12001) - fl207p produces: 



Wy^[2aYiX,-X) -{¥,-¥)] ; 



— = -WyXXi-X) ; 



(208a) 
(208b) 



day 



[Wy)20 



(209) 



dXi 

dijj 



ay 



(wj/)oo 



+ 



+ 



X 



2ay{X, - X) - {Yi - Y) 



X 



_{Wy)00 {Wy) 



-{X,-X) 



20 



(210a) 
(210b) 



and the substitution of Eqs. fl208bp . 



and fl2T0bl) into ([70]) and ^ 
yields an explicit expression of the regression line slope and intercept vari- 
ance estimator, {(t'^^Y ^'^d i^'i respectively, with no account taken of the 
scatter of the data points, Pj = (Xj,Fj), about the regression line, Eqs. f[72]) 
and ([73]). 
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In the special case of errors in Y negligible with respect to errors in X, 
analysed in subsection ESI Eqs. (dM]), (HMD, (HIS]), (HBID, (HSHD, (HSDD, via 
Eqs. (188])-([9I]) reduce to: 

w _ " 

W^ = —^ ; (wa;)oo = I^w^. = ; (211) 

U: = ■ (212) 



w. 



K-^; (213) 

U:' = ■ (214) 

f/r = ; (215) 

P: = ; (216) 



and the particularization of Eq. ( 135|) to the case under discussion via Eqs. ( 188|1 - 
(19T]) . selecting Q = V^, t/, -P; p = 0, 1; g = 1, 0; and using Eq. fl25|) . yields again 
Eqs. ([206]) and flW]) . Finally, the particularization of Eqs. (fTMj) . f lT95|) . firQQjl . 
to the case under discussion via Eqs. (120^]) . (EUTj) . and 

(Em)- (EH produces: 

{dx)-'w,XY^-Y) ■ (217a) 



■^ = {ax)'^w,AMX^-X)-2{Y,-Y)] ; (217b) 



ax)-'(5rjo2 ; (218) 



ddx 



dip 
dXi 

dip 



dxX 



[WxjOO 



+ 



2{Y,-Y)-dxiXi-X) 



(219a) 
(219b) 



and the substitution of Eqs. (l^TTa]) . (^m . and fl^M into and 
yields an explicit expression of the regression line slope and intercept vari- 
ance estimator, ((T^^)^ and i^'i^Y: respectively, with no account taken of the 
scatter of the data points, Pj = {Xi,Yi), about the regression line, Eqs. (I97|) 
and m^. 

In the special case of generalized orthogonal regression, analysed in sub- 
section EZl Eqs. (dHOD, (dUD, dUSD, (HHID, dHHD, ([I90D, via Eqs. ffTT0|) - ffTT3l) 
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reduce to: 



w 

w[ = —^- (5r,)oo = E^-. = ; (220) 

rcw^. 

ui = , " ; (221) 



(a^ + — 2rca) 



^i=T^r—T^ ; (222) 



2 2 

r c 



U- = ,. , ; (223) 

3 

77'" = "^^^ ■ f224) 

(a2 + c2 - 2rca)3 ' ^ ^ 

^' " (a2 + c2 - 2rca)3 ' ^^^^^ 

and the particularization of Eq. (135 p to the case under discussion via Eqs. (IllOl) - 
ffml) . selecting Q = V, t/, P; p = 0, 1; g = 1,0; and using Eq. ([25]), yields 
again Eqs. flM and fl2U7D . Finally, the particularization of Eqs. (HMD, dUSl), 
ffTO . to the case under discussion via Eqs. flTTUD- ffTT^ . fl2II5|) . f l2U7D . and 
fl220D-fl225D produces: 

90 W7.j(ac)' - c2](y. - r) + 2w,^cac{c - rac)(X, - X) 



dXi [(ac)^+ c2 - 2rcac]2 

d(j) _ w^^[{acf - c^]{Xi -X)- 2w^X'^c - rc){Y, - Y) 

dVi [(ac)^ + c2 - 2rcac]2 

= [{acf + c^- 2rcac]~^{2[rc{w,)2o - {w,)n]{ac? 



; (226a) 
(226b) 



dac 

+ 2[c\w,)i,- rc{w,)o2] [Sac - 2rc] } ; (227) 



dip acWxi Xw^ 



-1 



dXi (wx)oo [{acY + - 2rcacY \dac ) 

X {[{acf - c\Yi -Y)- 2cac[rac - c](X, - X)} ; (228a) 

dj) _ Wx, Xw^^ / 

(w;)oo [(oc)^ + c2 - 2rcac]^ V'^ac / 

x{[(ac)'-c2](X,-X)-2[ac-rc](F,-F)} ; (228b) 
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and the substitution of Eqs. fl22B]) . (1277]) and fl22g]) into f fTTB]) and ffTTTjl yields 
an explicit expression of the regression line slope and intercept variance esti- 
mator, {cr'a^Y and respectively, with no account taken of the scatter 
of the data points, Pj = about the regression line. 

In the limit of uncorrelated errors, r 0, Eqs. fl2^ . (12771) and fl2^ 
reduce to: 

"^^^ {[{acf - c\Y, -Y) + 2acc\X, - X)} ; (229a) 



dX, [(ac)2 + c2]2 



{[(ac)'-c2](X,-X)-2ac(r,-r)} ; (229b) 

(230) 



ac/i [3(ac)' - c2][(^ir,)o2 - c\w^)2o\ - 2ac[{acf - ?>c^]{w^)ii 



dac [(ac)2 + c^]^ 

X[(ac)' + 

(231a) 



[(ac)2 - c>,,(F, - F) + 2accV,(X, - X) 



[3(ac)2 - c2][(t(;^.)o2 - c'^{w^)2o] 


-2ac[(ac)2 


- 3c2](w^.)n 


^[(«c)^ + c^] 






x)00 






[(ac)2-c>.,(X,-X) 


- 2acWx,(>^i 





c?>^: (w.T._„ 

[jacr - c>.,(X, - X) - 2ac^.,(y. - Y) 
" [3(ac)2 - c^][{w^)o2 - c^{w^)2o] - 2ac[(ac)2 - 3c^]{w,)n' ^ ' 
on the other hand, Eq. (11191) may be cast under the equivalent form: 

[{acY - c^]{wx)u = ac[{wx)o2 - c'^{wx)2o] ; (232) 
and the substitution of Eq. fl2^ into fl2^ and fl2^ yields: 

dcf) {wx)ii 



dac ac[(ac)^ + c2] 

dip _ acWx, _ Xac 
dXi iwx)oo (ac)^ + 

^ [{acf - c>.,(y^ -Y) + 2acc^Wx^{X, - X) 

dip _ Xac 

dYi (mr)oo {acy + 



(233) 



(234a) 



^ [(ac)^ - c>,,(X. -X) - 2dcWxAY. - Y) 
(wx)ii 
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finally, the substitution of Eqs. fl22^ . fl25^ and ^Ml into ffTTB]) and ffTTTj) 
particularized to uncorrelated errors, r = 0, yields an explicit expression of 
the regression line slope and intercept variance estimator, and (<5"^^)^, 

respectively, with no account taken of the scatter of the data points, Pj = 
{Xi,Yi), about the regression line, Eqs. (11281) and (11291) . 



C Regression line slope and intercept vari- 
ance estimators 

According to Eqs. (l46l) and (H7|) . an explicit expression of the squared 
residual trace implies an explicit expression of the regression line slope and 
intercept variance estimators, ((3"a)^ and (o"^)^, respectively (Y69). The former 
may be obtained by substitution of Eqs. (fT^ and (l?T]) into (ITT]) . 

After some algebra, the result is: 



i=i ^ 



1 W,^{aXi+b-Y^^ 1 



X [{aVti - nf + {aVt,n - 1)' - 2n{aVti - ri){aVtiri - 1)] ; (235) 

where, in addition, the following relation holds: 

{aVti - r-if + {aVtiri - if - 2ri{aQi - ri){aQiri - 1) 

= [{afn^ - 2aar, + 1](1 - rf) = w^^^W'^l - rf) ; (236) 

where the former equality makes an identity and the latter is owing to 
The substitution of Eq. (12361) into (12351) yields: 

TJi = f2W^{aX, + b-Y,y ; (237) 

i=l 

and the combination of Eqs. (I27|) . (I35l) and (12371) produces: 

T^= {dfW2o + Wo2-2aWn ; (238) 

which is the result of interest. 

The substitution Eqs. (gO]), (flMD . (11951) and (l238|) into dH]) yields an 
explicit expression of the regression line slope variance estimator, (o"a)^, with 
due account taken of the scatter of the data points, Pj = (Xj, Yi), about the 
regression line (Y69). 
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The substitution of Eqs. ffTO . and (12551) into (US]) yields an explicit 
expression of the regression line intercept variance estimator, ((3"g)^, with due 
account taken of the scatter of the data points, Pj = {Xi,Yi), about the 
regression line (Y69). 

If Eqs. (14T]) and fH5|) are approximated by Eqs. fH6i) and fH7|) . respectively, 
the result is expressed by Eqs. (jlH]) and (H^ . respectively. 

In the special case of unweighted residuals, Wi = W, Wpq = WSpg, 
Eq. fl238|) reduces to: 



in terms of deviation traces. 



ayS2o + ^02 - 2aSn ; (239) 



D Equivalence between earlier and current 
formulation 



In the special case of homoscedastic functional or extreme structural mod- 
els, where the ratio of instrumental or intrinsic dispersions in the two vari- 
ables, Y and X, maintains constant, {o'yy)i/{axx)i = c^, 1 < < and the 
errors in Y and in X are uncorrelated, ri = 0, 1 < i < n, the regression line 
slope estimator and the related dispersion estimator read: 



>S'n9 — c^So 




1 + 



20 



2Sn 



1/2- 



5'o2 — aSii \ 

^ 2 ^ 



{af 



Sm — clS 



11 



n 



R 



n 



1 " 



(240) 

(241) 
(242) 



under the further assumptions of unweighted normal residuals and large sam- 
ples (FB92). With regard to Eq. fl240p . the plus instead of the double sign 
appears in the parent paper (FB92) but the latter is mentioned in an earlier 
paper (Ial90). In this case, the (physically meaningless) parasite solution 
must be disregarded. 

The substitution of Eqs. (IHTl) into (1242!) yields after some algebra: 



R 



n-2 

in terms of deviation traces. 



5*02 + {afS2Q - 2aSu 



(243) 
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The substitution of Eq. mUh into (12151) yields: 

1 T~ 

R = — ^ ; (244) 

n-2W ' ^ ' 

which shows the relation between different sums of squared residuals, R and 
Tg, via the dimensional constant, — 2)iy]. A dimensionless counterpart 
to Eq. IM^ reads: 

= — ; 245 
aSii n-2 aWn 

where Wu = WSn in the case under consideration. 

In the limit of errors in X negligible with respect to errors in Y, — )■ +oo, 
the square root on the right-hand side of Eq. (12401) may be developed in 
binomial series with the terms of higher order neglected. The result is: 

aY = fi ; (246) 

'-'20 

where the plus in the double sign on the right-hand side of Eq. (12401) corre- 
sponds to a (physically meaningless) infinite value and for this reason has 
been disregarded, while the minus has been considered. 

The combination of Eqs. (1241 p and (I243p . after neglecting the terms of 
higher order with respect to unity, yields: 



ay 



and the substitution of Eq. (1246 p into (I247p produces: 

h 



So2 + {avYS^o - 2aYSn ; (247) 



n - 2 (5n)2 



(248) 



in terms of the deviation determinant, expressed by Eq. (I57p . It can be seen 
that Eqs. (1246 p and (I248p coincide with ( 15^ and (15^ . respectively, which 
implies the equivalence between earlier (FB92) and current (this paper) for- 
mulation, in the special case under discussion (c^ — > +oo). 

In the limit of errors in Y negligible with respect to errors in X, — )■ 0, 
the term in square brackets on the right-hand side of Eq. ( 1240 p tends to unity. 
The result is: 

ax = |^ ; (249) 

where the minus in the double sign on the right-hand side of Eq. (12401) corre- 
sponds to a (physically meaningless) null value and for this reason has been 
disregarded, while the plus has been considered. 
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With this restriction, the square root on the right-hand side of Eq. fl240p 
may be developed in binomial series with the terms of higher order neglected. 
After some algebra, the result is: 



'S'02 — axS 



11 



in terms of the deviation determinant, Eq. ( 1S7|) . 
The substitution of Eq. fl250D into flMTD yields: 



(250) 



11/ 



[ax 



ax 



Si 



02 



n - 1 (5, 



02 J 



(251) 



and the substitution of Eqs. (1243 p and (12491) into (12511) produces: 



n 



2{S 



11} 



1 



n 



1{S 



11) 



(252) 



in terms of the deviation determinant, expressed by Eq. fl57|) . It can be seen 
that Eq. (I249p coincides with Eq. (I104p . which implies the equivalence be- 
tween earlier (FB92) and current (this paper) formulation, in the special 
case under discussion (c^ — )■ 0), concerning the regression line slope estima- 
tor, ax. The contrary holds for the regression line slope variance estimator. 



((Jax)^, where Eq. (I252p overstimates Eq. (llOSp . the difference decreasing for 
increasing n and/or decreasing Do,/ {SuY , and tends to be null in the limit, 
n +00 and/or Ds/{SiiY 0. 

Turning to the general case, it can be seen that Eq. (I120p for unweighted 
residuals, Wx, = Wx, {wx)pq = uixSpq, reduces to Eq. (I240p . which implies the 
validity of the pseudo quadratic, Eq. (I119p . The substitution of Eqs. (I134p 
and Eq. (1243 p into (I24ip yields after a lot of algebra: 



n 



20 



n 



1 V Sn 



- 1 



(253) 



where in the limit, c — )• +00, c — )■ 0, Eq. 



reduce to fl21g]) and fl25^ . 
respectively. A comparison between Eqs. (I253p and (I142p shows that earlier 
(FB92) and current (this paper) formulation are different due to the second 
term within square brackets. 

In the special case, = S02/S20, which implies a = {802/820)^^'^, a similar 
conclusion is attained by comparison between Eqs. (I253p and (11540 . 
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Figure 1: Regression lines related to [0/H]-[Fe/H] empirical relations de- 
duced from two samples with heteroscedastic data, RB09 and Sal09, and 
three samples with homoscedastic data (using the computer code for het- 
eroscedastic data), Fal09, cases LTE, SHO, and SHI, indicated on each panel 
together with related population and model captions. The regression lines 
related to five different methods are shown for each sample on the top right 
panel. For further details refer to the text. 



55 




Figure 2: Regression lines related to [0/H]-[Fe/H] empirical relations de- 
duced from two samples with heteroscedastic data (with instrumental scat- 
ters taken equal to related typical values), RB09 and Sal09, and three samples 
with homoscedastic data, Fal09, cases LTE, SHO, and SHI, indicated on each 
panel together with related population and model captions. The regression 
lines related to four different methods are shown for each sample on the top 
right panel. For further details refer to the text. 
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